Thanks for testing.

I guess this bug can finally be closed now (after almost 13 years).
I'm not sure if this is something to celebrate or to mourn about. 😉
That made it the oldest report that was still open afaict.

Closing as fixed in Octave 9.

Yes I saw the related discussion elsewhere. I'll prepare a new crossbuild later this week and try again.

this bug is still present in octave 8.0.1 (hg id: 24bd675bceab)

testing on a ntfs local ssd in windows 10:

>> cd d:
>> edit newtestfile (create a new script file and save it)
>> newtestfile
error: 'newtestfile' undefined near line 1, column 1
>> ls -l
total 30
<snip>
drwxr-xr-x 1 Admin None   0 Jan 10  2022 '$RECYCLE.BIN'
drwxr-xr-x 1 Admin None   0 Nov  9 01:02  MapData
drwxr-xr-x 1 Admin None   0 Mar 22  2022  Programs
drwxr-xr-x 1 Admin None   0 Aug  9  2020  Recovery
drwxr-xr-x 1 Admin None   0 Aug  9  2020 'System Volume Information'
drwxr-xr-x 1 Admin None   0 Mar 21  2022  localdata
-rw-r--r-- 1 Admin None   9 Dec  8 18:21  newtestfile.m
drwxr-xr-x 1 Admin None   0 Jan 10  2022  temp
>> newtestfile
error: 'newtestfile' undefined near line 1, column 1

also, trying Markus's stat test, things behave fine in a subfolder:

>> cd temp
>> dir
.   ..
>> stat('.')
ans =

  scalar structure containing the fields:

    dev = 3
    ino = 0
    mode = 16895
    modestr = drwxrwxrwx
    nlink = 1
    uid = 0
    gid = 0
    rdev = 3
    size = 0
    atime = 1670543181
    mtime = 1641824642
    ctime = 1590511359
    blksize = NaN
    blocks = NaN

>> fclose(fopen('testfile.m','w'))
ans = 0
>> stat('.')
ans =

  scalar structure containing the fields:

    dev = 3
    ino = 0
    mode = 16895
    modestr = drwxrwxrwx
    nlink = 1
    uid = 0
    gid = 0
    rdev = 3
    size = 0
    atime = 1670543207
    mtime = 1670543207
    ctime = 1590511359
    blksize = NaN
    blocks = NaN

but in the root of the drive:

>> stat('.')
ans =

  scalar structure containing the fields:

    dev = 3
    ino = 0
    mode = 16895
    modestr = drwxrwxrwx
    nlink = 1
    uid = 0
    gid = 0
    rdev = 3
    size = 0
    atime = 315550800
    mtime = 315550800
    ctime = 315550800
    blksize = NaN
    blocks = NaN

>> fclose(fopen('testfile.m','w'))
ans = 0
>> stat('.')
ans =

  scalar structure containing the fields:

    dev = 3
    ino = 0
    mode = 16895
    modestr = drwxrwxrwx
    nlink = 1
    uid = 0
    gid = 0
    rdev = 3
    size = 0
    atime = 315550800
    mtime = 315550800
    ctime = 315550800
    blksize = NaN
    blocks = NaN

>> ls -l
total 30
<snip>
drwxr-xr-x 1 Admin None   0 Jan 10  2022 '$RECYCLE.BIN'
drwxr-xr-x 1 Admin None   0 Nov  9 01:02  MapData
drwxr-xr-x 1 Admin None   0 Mar 22  2022  Programs
drwxr-xr-x 1 Admin None   0 Aug  9  2020  Recovery
drwxr-xr-x 1 Admin None   0 Aug  9  2020 'System Volume Information'
drwxr-xr-x 1 Admin None   0 Mar 21  2022  localdata
-rw-r--r-- 1 Admin None   0 Dec  8 18:31  newtestfile.m
drwxr-xr-x 1 Admin None   0 Dec  8 18:46  temp
-rw-r--r-- 1 Admin None   0 Dec  8 18:52  testfile.m
>> testfile.m
error: 'testfile' undefined near line 1, column 1

are those time values normal in a root folder?

Sorry bad read, I thought I had to write the file with the instruction, here is the correct test (I think). The data drive is an additional SSD disk.

>> format long
>> which test1
>> stat('.')

ans =

  scalar structure containing the fields:

    dev = 3
    ino = 0
    mode = 16895
    modestr = drwxrwxrwx
    nlink = 1
    uid = 0
    gid = 0
    rdev = 3
    size = 0
    atime = 1635509742
    mtime = 1635509742
    ctime = 1635509742
    blksize = NaN
    blocks = NaN

>> fclose(fopen('test1.m', 'w'));
>> stat('.')

ans =

  scalar structure containing the fields:

    dev = 3
    ino = 0
    mode = 16895
    modestr = drwxrwxrwx
    nlink = 1
    uid = 0
    gid = 0
    rdev = 3
    size = 0
    atime = 1635509742
    mtime = 1635509742
    ctime = 1635509742
    blksize = NaN
    blocks = NaN

>> which test1

Hi again:

It must be a problem with the disk drive (D:), I have repeated the process on home disk (C:) and there are no problems....

Octave is installed in home disk (c:) as usual, but data are in data disk (d:).

I will check the data disk (d:) configuration....

Re-starting Octave isn't the solution. I just wanted to see if it is something intrinsic to that file or folder...

Re-reading the discussion, it looks like rehash uses some logic to only re-scan the folders of which it thinks that they have changed.
Does re-creating the load path cache completely with `path(path())` help? (I hope that doesn't use some clever tricks to avoid rehashing paths it thinks it already knows...)

Additionally, IIUC, Mike's suspicion was that the modification date of the parent directory doesn't change. Could you try something like the following in a directory that doesn't contain a 'test1.m' file and show the results here?

format long
which test1
stat('.')
fclose(fopen('test1.m', 'w'));
stat('.')
which test1

Do these files reside on a local drive or a network share?
ISTR that there were missing signals for file updates from UNC shares (maybe also others).

Is the file found on a restart of Octave? Does issuing `rehash()` to manually trigger a load path cache update make a difference?

@paredes: Could you please show the exact error message or output you are seeing when calling your script doesn't work?

@Silvestre - just for reference all of the things that might be relevant: are you working on a local drive, a network drive, are you running Windows and which version, and if so are you in a Onedrive managed folder?

Closing as fixed as per comment #34.
If this behaviour should re-appear, we can re-open this bug or file a new one.

I cannot recreate it with Octave-5.0.0 on Windows 7 and on Windows 10.
In fact it works fine here even with octave-4.2.2 on Windows 10.

Just tried the following on Windows 10:

```

>> cd d:\        % toto.m does not exist yet
>> edit toto.m   % create script containing: disp hello
>> toto

```

As reported below, this fails in Octave 4.0 and 4.2 with error 'toto' undefined.

The good news is, it works as expected with Octave 4.4.

Unless someone is able to reproduce the problem with Octave 4.4 on some other version of Windows, this bug can be closed ;-)

I like Abhinav's suggestion of only forcing a rehash of a directory known to have changed.  I wouldn't call that a sledgehammer at all.

One way to make it more fine-grained would be to allow the second argument to be a filename rather than a directory.  Then adding a file is very efficient -- presumably much more efficient than doing a rehash of the whole directory based on a timestamp.  The only need for an inefficient whole-directory rehash is that a file may have been added by a program external to (and not triggered by) Octave.

Also, I think you would be looking for "FindFirstChangeNotification", the Windows equivalent to Linux's inotify.  I don't think MinGW supports inotyify.  http://stackoverflow.com/questions/36125844/use-inotify-with-mingw

@Mike: Adding a rehash command doesn't sound all that bad to me.  Certain system shells, like csh, have a rehash command to update internal caches after you have changed the PATH variable.

The root cause seems to be known which is that Octave relies on timestamps, and timestamps on Windows filesystems are not always updated.  I don't think more detective work will help.

If we want to fix the root cause, then I like your suggestion of looking at inotify.  I've never done anything with it though so I can't say how hard it would be.

rehash seems like a nice workaround if we don't want to address the root cause.

To solve this issue, i propose to add a new function 'force_rehash(dir_name)' to force a rehash of the dir without checking for timestamp or edit the current rehash so that 'rehash' or 'rehash(false)'  remain as is. But 'rehash(true, dir_name)' could be used to force rehash irrespective of timestamp!
.
This won't be a huge change and will allow users to force a rehash on windows and certainly solve this issue.
.
Should I do the changes and submit a cset? Does anyone have a problem with this approach (just asking as the issue is old and it hasn't been implemented yet)...

Normally, in root dirs there shouldn't be so awfully many files.

I don't know Octave's function/script file loading internals, so apologies if the following is a dumb question; but I wonder, why stat all files? all that seems to be needed is to just stat the called function file in question, and then only once after entering each Octave command (as changing some called function file contents while Octave is accessing it in e.g. a loop wouldn't be a good idea.)

So if this needs to be fixed at all, maybe modding the function/script load logic for Windows systems is the way to go?

I agree with Rik that this can get quite messy.
NTFS isn't a monolithic static file system; there are several versions around. M$ also still has the WinFS looming somewhere, possibly with still other behavior.
On NTFS it is possible to map other drives somewhere in the tree a la *nix mount (on Windows it's called junctions IIRC); we would need to know how those behave as well: as "local" subdirs, or do they behave as remote "root" dirs? in the latter case we'd also need to detect them as such.


Even while this bug is quite old is hasn't seen many confirmations; so I'd also agree with lowering priority/severity until we get more requests.
Repeating myself (see comment #13), having script or function files in root dirs is something that can be avoided.

- IMO on local fixed drives (HDs) it's a bad idea anyway.

- Only on thumb drives and other removable drives, and in some cases on mapped network drives, it can be more reasonable to keep m-files in the root dir.

Perhaps there is a way to detect a directory to be the root of a removable/network device and act accordingly along Jordi's suggestion.
Note that thumb drives, and to a lesser extent network drives, are usually slow; statting files may give a noticeable performance hit.
(I don't know if mapped network drives behave the same as "physical local" drives as regards directory time stamps.)

Yes, confirmed here (I now have a winxp VM with 3.8.0 to test with). The root directory of an NTFS drive seems to have a constant timestamp:

octave:1> format long
octave:2> info = stat ('C:\')
...
    atime =  315550800
    mtime =  315550800
    ctime =  315550800

octave:3> ctime (315550800)
ans = Tue Jan 01 00:00:00 1980
octave:4> edit ('C:\foo.m')
octave:5> info = stat ('C:\')
...
    atime =  315550800
    mtime =  315550800
    ctime =  315550800

So yes this looks unavoidable given the current path caching scheme.

One possibility might be to watch for directory inode changes instead of looking at timestamps. This could use FindFirstChangeNotification on Windows and inotify on Linux-based systems. This would most likely be a major overhaul.

Re-reading comments #7 and #8, you'll see that this is primarily about the directory timestamp, not necessarily the file timestamp. So Octave is expecting the timestamp of the containing directory to be updated. Do drive top-level directories have timestamps in Windows, and are they updated when new files are created at the top level?

What does

format long
info = stat ('X:\')

show before and after creating a new m-file?

Sorry, I need to be a bit more exact:

do:
"start Octave"
(or "start octave-cli")
.... from that same cmd32 window.

FYI, "start octave" will detach an octave session from that command window. While you're fiddling in Octave, from that cmd32 window you can run "touch".

============
Anyway I've tried this myself with Octave-3.9.0+ and I get the same as the OP. Screen shot of session attached, just to show.
Like the OP I tried it on an NTFS drive (X: in my case)
"touch"ing the file (after Octave command #7-2) doesn't help.

BTW I'm intrigued by the error msg:

octave:7> type test2.m
error: type: couldn't open '' for reading

i.e. Octave seems to forget the file name although it was entered on the command line.

The command touch does not exist on Windows

A substitute to this command is : copy /b file +,,
(found on http://stackoverflow.com/questions/51435/windows-version-of-the-unix-touch-command), but it doesn't solve the problem.

Yes I'm following but a bit short of time lately - maybe tonight or otherwise tomorrow I'll check. & I also have to (re)build 3.8.0 first.

From what I've read so far I inferred that Octave relies on subdirectory timestamps, not individual function-/script-file timestamps.
Very probably -if not undoubtedly- for good reason (not hard to guess); but a drawback is this sort of issues.

A workaround is to avoid root dirs. Even on thumb drives (USB sticks) that would be a good way.

I just reformatted a USB memory stick to use NTFS instead of the default FAT32 and Octave works perfectly with it.  So the diagnosis about timestamps is accurate.

John, I was able to check the file system type by double-clicking "My Computer" to get a list of disk drives.  If I right-click on a disk drive and then click on Properties it brings up a new tabbed window.  In the General tab there is an entry called File System.

No, sorry, the problem also occurs on NTFS partitions ! So it's not a good idea to lower the priority

Very definitely confirmed.  Octave seems to be caching the contents of a directory on start-up and then never updating the cache.  This could be down to differences in how the operating system updates the inode timestamp for the directory.

On Linux, creating a new file in a directory updates the timestamp.  I think Octave is comparing the current directory timestamp against the cached timestamp to figure out if it needs to go and re-read the directory.

On Windows, creating a new file in a directory does not change the timestamp--I know because I tried.  However, I have the UNIX tool touch.exe installed and when I touch the directory to bring the timestamp up to the current time then Octave correctly finds the new m-files.

This sequence worked:

cd e:\
mkdir tmptst
cd tmptst
edit junk.m
## Add some code to junk.m in the editor and then quit
which junk.m
## Returns nothing
## Now, in another shell window execute touch e:\tmptst
which junk.m
'junk.m' is the file e:\tmptst\junk.m

Seems like it might require a lot of O/S specific code to solve this one.

Thanks for reminding me, I'll check it out.

[Little bit OT]
A similar problem I hit with recent dev versions (3.7.7 & 3.9.0 and the two 3.8.0 release candidates) is that editing/debugging functions in private/ dirs leads to similar behavior (on MinGW):
Octave suddenly doesn't seem to be able to find certain private/functions anymore (while other private/ functions weren't affected). Adding the private subdit to the load path (as the GUI editor helpfully suggests) didn't help. Only a restart helps.
But as this behavior was erratic, and because I don't know the exact rules for the search path as regards private subdirs, I thought I was probably fubarring up myself so I didn't file a bug report.
After all, things work OK as long as functions in private subdirs aren't touched.

Octave 3.7.7+ MXE GUI built for MinGW, installed on drive X, and Octave-3.6.7+ MXE CLI (ditto):

With either script or function files, it doesn't even work in a subdir of root:

octave-cli.exe:1> pwd
ans = X:\Octave\octave-3.7.6+_MXE
octave-cli.exe:2> cd f:/tmp
octave-cli.exe:3> edit script.m
octave-cli.exe:5> dir script.m
script.m
octave-cli.exe:6> script
error: 'script' undefined near line 1 column 1
octave-cli.exe:6> rehash
octave-cli.exe:7> script
error: 'script' undefined near line 1 column 1
octave-cli.exe:7> clear -f
octave-cli.exe:8> script
error: 'script' undefined near line 1 column 1
octave-cli.exe:8>
:
:
octave-cli.exe:8> cd /
octave-cli.exe:9> pwd
ans = f:\
octave-cli.exe:10> edit script2.m
octave-cli.exe:11> dir *.m
script2.m
octave-cli.exe:12> script2
error: 'script2' undefined near line 1 column 1
octave-cli.exe:12>

After restarting, Octave behaves even stranger:

octave-cli.exe:1> cd f:\tmp
octave-cli.exe:2> dir *.m
funcfil.m                  strread_bitw_regexp.m      textread.m
script.m                   strread_bitwidth.m         textscan.m
strread.m                  strread_bitwidth_regexp.m
octave-cli.exe:3> funcfil    ## Works OK
Hello world!

octave-cli.exe:4> script     ## does nothing
octave-cli.exe:5> type script.m
script.m is the user-defined function defined from: .\script.m

disp ("Hello World!\n")';

octave-cli.exe:6>
octave-cli.exe:7> cd /
octave-cli.exe:8> dir *.m
funcfil2.m  script2.m
octave-cli.exe:9> script2    ## Works OK;
Script 2

octave-cli.exe:10> funcfil2
octave-cli.exe:11> funcfil2
octave-cli.exe:12> type funcfil2
funcfil2 is the user-defined function defined from: f:\funcfil2.m

function [retval] = funcfil2 ()

        disp ("Funcfil2\n")';

endfunction

octave-cli.exe:13> type script2
script2 is the user-defined function defined from: f:\script2.m

disp ("Script 2\n");

octave-cli.exe:15>which funcfil2
'funcfil2' is a function from the file f:\funcfil2.m

Striking is that it works upon restart for functions but not for scripts in a subdir, and the other way round in the root dir.

Could these be MinGW bugs (in the gcc runtime libraries)?

Still present in 3.6.0 MinGW