bugGNU Octave - Bugs: bug #62576, dir() behavior changed in octave...

 
 

bug #62576: dir() behavior changed in octave 7.1.0 for network drives

Submitter:  Carlos Fuentes <fito>
Submitted:  Fri 03 Jun 2022 03:51:13 PM UTC
   
 
Category:  Octave Function Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Incorrect Result
Status:  Fixed Assigned to:  None
Originator Name:  dir Open/Closed:  * Closed
Release:  * 7.1.0 Operating System:  * Microsoft Windows
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Tue 05 Jul 2022 03:40:19 PM UTC, comment #15: 

Worked for me in my (limited) testing.

If there are still issues, we can re-open - or start with a fresh report.

Closing as fixed.

Markus Mützel <mmuetzel>
Group administrator
Tue 05 Jul 2022 03:08:19 PM UTC, comment #14: 

No further comments for 2 weeks.  Okay to close report as Fixed?

Rik <rik5>
Group administrator
Mon 20 Jun 2022 05:39:53 PM UTC, comment #13: 

Thanks for raising that.
Even if Octave now supports UNC paths (mostly), third party software might not.
I pushed a change for `canonicalize_file_name` that reverts the conversion from mapped network drive to UNC path (that Windows does automatically):
https://hg.savannah.gnu.org/hgweb/octave/rev/b47bf773c508

Re-opening and marking as ready for test again.

Markus Mützel <mmuetzel>
Group administrator
Mon 20 Jun 2022 03:00:19 PM UTC, comment #12: 

just noting that in some older versions of either Octave or Windows (I can't remember specifically which) CMD cannot handle UNC paths. This used to be a problem in Octave and I had mapped certain network shares to drive letters to get around this. The expectation was that once something was mapped to a drive letter one would no longer see the UNC path under any typical usage.

issue highlighted here (but not really explaining which versions it's relevant in):
https://stackoverflow.com/questions/45071585/cmd-does-not-support-unc-paths-as-current-directories

Nicholas Jankowski <nrjank>
Group Member
Mon 20 Jun 2022 02:50:57 PM UTC, comment #11: 

Closing as fixed.

Markus Mützel <mmuetzel>
Group administrator
Mon 20 Jun 2022 02:50:34 PM UTC, comment #10: 

Thanks for checking again.

For future reference (to self): It is not `__wglob__` that has changed behavior with mapped network drives (like I erroneously claimed before). But it is `canonicalize_file_name` which returns a UNC path now.

That was probably this changeset:
https://hg.savannah.gnu.org/hgweb/octave/rev/d7223245d26f

Markus Mützel <mmuetzel>
Group administrator
Mon 20 Jun 2022 02:45:57 PM UTC, comment #9: 

I am sorry, I thought that the fact that it was a UNC path would automatically break the code as it did with 7.1.0.
When I saw it after the bugfix, I thought it will keep generating issues (as in the 6.4.0 the old format was in place)

I tried all, including dir() and load() and csvread() and everything works, its just that the path is UNC instead of root drive letter format. I apologize, I guess need to drink more coffee and learn a a bit more.

To summarize, the issue is fixed. I apologize for the extra trouble but I thank you very much for the time and support with the issue, and for teching me a thing or two.

Thank you very much! Regards!


comment #8:

> Re-reading: Does `load` and `csvread` not work with UNC paths?

Carlos Fuentes <fito>
Mon 20 Jun 2022 02:24:36 PM UTC, comment #8: 

Re-reading: Does `load` and `csvread` not work with UNC paths?

Markus Mützel <mmuetzel>
Group administrator
Mon 20 Jun 2022 02:23:10 PM UTC, comment #7: 

Do you know why that doesn't work? Paths starting with a drive root and UNC paths should be equally valid as paths in Windows...

Do you always call `dir()` without input argument? In that case, you could probably replace `a(k).folder` by `pwd()`.

Markus Mützel <mmuetzel>
Group administrator
Mon 20 Jun 2022 01:35:58 PM UTC, comment #6: 

I apologize for the imprecision. It provides the server and the share, my example was oversimplified.

It provides


ans = \\The_network_drive\share


I am not versed in the underlying mechanisms of dir(), but I know this new behavior causes problems for us because we had a crawler code that when run used dir() to move around in folder structures, find files and process them while building the paths using this core script idea in a recursive manner:


a = dir();

if k<=2; error('folder is empty'); end;

for k = 3:length(a)
   path = [a(k).folder '\' a(k).name];
   process_file(path);
end


however, when trying to use that path and '\\server\share' is there instead of 'Z:\', the code does not work. Of course as workaround the '\\server\share' can be replaced automatically. Another option is to reference everything relative to the current folder, but the issue remains. Creating an absolute address that a command such as load() or csvread() can use is not possible using the output of dir and constructing a path with it ("path = [a(k).folder '\' a(k).name]").

Thank you for your time and considering this issue

comment #4:

> Does it really just return the server? And not the share?
>
> IIUC, Windows is returning `\\server\share\file` for files on network shares for `FindFirstFileW` and `FindNextFileW`. `__wglob__` (the underlying work horse for the `dir` function on Windows) is using these functions to be able to handle non-ASCII characters in paths. That'll probably stay that way.
>
> Is that causing problems for you?
>

Carlos Fuentes <fito>
Mon 20 Jun 2022 12:31:18 PM UTC, comment #5: 

Forgot to mention: `FindFirstFileW` and `FindNextFileW` don't seem to care whether a network share was mapped to a drive letter or not...

Markus Mützel <mmuetzel>
Group administrator
Mon 20 Jun 2022 12:30:15 PM UTC, comment #4: 

Does it really just return the server? And not the share?

IIUC, Windows is returning `\\server\share\file` for files on network shares for `FindFirstFileW` and `FindNextFileW`. `__wglob__` (the underlying work horse for the `dir` function on Windows) is using these functions to be able to handle non-ASCII characters in paths. That'll probably stay that way.

Is that causing problems for you?

Markus Mützel <mmuetzel>
Group administrator
Mon 20 Jun 2022 12:22:42 PM UTC, comment #3: 

Hello, sorry that I took a while to test. I tested 7.1.1 (installed octave-2022-06-10-00-11-release-w64-installer) and the problem persist when using network drives: I get the '\\the_network_drive\' instead of 'Z:\' when evaluating

Example: being in 'z:\' drive

+Verbatim+

>a = dir()
>a(1).folder:


\\the_network_drive\

-Verbatim-

comment #2:

> Forgot to mention: You can test if this is working for you now with a nightly build from here:
> https://buildbot.octave.space/#/download



Carlos Fuentes <fito>
Sun 12 Jun 2022 02:54:21 PM UTC, comment #2: 

Forgot to mention: You can test if this is working for you now with a nightly build from here:
https://buildbot.octave.space/#/download

Markus Mützel <mmuetzel>
Group administrator
Fri 03 Jun 2022 10:15:20 PM UTC, comment #1: 

Thanks for reporting.

Unfortunately, some things broke with the new implementation of an Unicode aware `__wglob__` on Windows.

UNC paths should be working again after this change:
https://hg.savannah.gnu.org/hgweb/octave/rev/d33af4594440

Marking as ready for test.

Markus Mützel <mmuetzel>
Group administrator
Fri 03 Jun 2022 03:51:13 PM UTC, original submission:  

Hello,

I am reporting a bug, as I noticed the behavior of dir() changed from 6.4.0 to 7.1.0. Lets say we have a network drive called '\\The_network_drive\' mounted in 'Z:\'. And there is one folder in it, called 'The_folder' (Z:\The_folder), which has one file 'The_file.txt'.

In 6.4.0, if we are standing in 'Z:\' and we write:

+Verbatim+

>> a = dir();
>> b = dir([a(3).folder '\' a(3).name]);


b =

 3x1 struct array containing the fields:

    name
    folder
    date
    bytes
    isdir
    datenum
    statinfo

-Verbatim-

we will get the contents of 'The_folder' in b, in a structure ('.', '..','The_file.txt')

However, in 7.1.0, if we do the same, it does not work. Because in 7.1.0, a(3).folder is not 'Z:\', but '\\the_network_drive\', and when doing b = dir('\\the_network_drive\The folder'), dir returns an empty structure.

+Verbatim+

>> a = dir();
>> b = dir([a(3).folder '\' a(3).name]);


b =

 0x1 struct array containing the fields:

    name
    folder
    date
    bytes
    isdir
    datenum
    statinfo

+Verbatim+

Following this logic, any recursive element or "folder crawler" we want to write based on dir() that works with network drives will have the issue of not being able to reference its absolute address correctly, or more specific, in a useful way.

Thank you for reading and considering the issue.

Running on Windows 10 21H2, with installer versions (x64) of both octave 6.4.0 and 7.1.0

Carlos Fuentes <fito>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by rik5 (Posted a comment)
  • -email is unavailable- added by nrjank (Posted a comment)
  • -email is unavailable- added by mmuetzel (Posted a comment)
  • -email is unavailable- added by fito (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 7 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2022-07-05 mmuetzel StatusReady For Test Fixed
        Open/ClosedOpen Closed
    2022-06-20 mmuetzel StatusFixed Ready For Test
        Open/ClosedClosed Open
    2022-06-20 mmuetzel StatusReady For Test Fixed
        Open/ClosedOpen Closed
    2022-06-03 mmuetzel StatusNone Ready For Test

    Back to the top

    Powered by Savane 3.13-4448.
    Corresponding source code