bugGNU Octave - Bugs: bug #59231, Incorrect result of...

 
 

bug #59231: Incorrect result of "dir" using wildcards with non-ASCII file names

Submitter:  Paryshaan <rouhollah>
Submitted:  Wed 07 Oct 2020 06:10:18 AM UTC
   
 
Category:  Octave Function Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Incorrect Result
Status:  Fixed Assigned to:  None
Originator Name:  Rouhollah Open/Closed:  * Closed
Release:  * dev Operating System:  * Microsoft Windows
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Sat 10 Oct 2020 12:31:32 PM UTC, comment #5: 

This works for me now with hg id 32dde927e59b (which will eventually be part of Octave 7).

Closing as fixed.

Markus Mützel <mmuetzel>
Group administrator
Thu 08 Oct 2020 02:19:48 PM UTC, comment #4: 

I pushed a change that should fix `dir` with non-ASCII filenames on default here:
https://hg.savannah.gnu.org/hgweb/octave/rev/4e64416772f1

While looking into this, I also noticed a regression with `stat` with non-ASCII filenames on Windows which I fixed on stable:
https://hg.savannah.gnu.org/hgweb/octave/rev/ed029f40be15

Marking as ready for test.

Markus Mützel <mmuetzel>
Group administrator
Thu 08 Oct 2020 09:06:24 AM UTC, comment #3: 

Unfortunately, that doesn't work on Windows with Octave 6.0.91 or the current default branch (Octave 7).

We are using the `glob` function provided by gnulib. Gnulib doesn't provide a Unicode interface on Windows. Unfortunately that means that input and output is limited to ASCII characters.

Since there is no one-to-one equivalent of `glob` on Windows, the only way I see is to use alternative functions from the WinAPI. Maybe, we could use `FindFirstFileW` which looks like it will support the wildcard patterns "?" and "*". And then build the list from calling `FindNextFileW` repeatedly.
Almost exactly what we are doing for `octave::sys::get_dirlist` already:
https://hg.savannah.gnu.org/hgweb/octave/file/500011ff7e3e/liboctave/system/lo-sysdep.cc#l103

It also looks like those functions don't support any other patterns (like "[ab]").
We already have `__wglob__` which takes that difference into account.
`dir.m` calls `__wglob__`. So, it would probably be reasonable to make the proposed change to `__wglob__` only.

I don't see a fix for `glob` that wouldn't mean reimplementing most of gnulib's `glob` which is probably out of Octave's scope. I'd suggest to add a note to the docstring of `glob` that mentions that it won't work for files with non-ASCII characters on Windows.
It's more important to have a working `dir` function imho.

Re-titling accordingly.

Markus Mützel <mmuetzel>
Group administrator
Wed 07 Oct 2020 08:30:55 PM UTC, comment #2: 

Running on Linux as well and this works for me too.  I think the glob routine is different between UNIX and Windows so it is possible that the Windows build of Octave does not handle this correctly.  It would be good if someone on Windows could check whether the current release candidate 6.0.91 runs this code correctly.

Rik <rik5>
Group administrator
Wed 07 Oct 2020 06:33:10 PM UTC, comment #1: 

Seems to be working with pre-release version (6.0.91) on linux, but make sure first that all your quotes match


octave:1> fl = dir('*.csv')
fl =

  scalar structure containing the fields:

    name = دما.csv
    folder = /home/dima/scratch/دما.csv
    date = 07-Oct-2020 14:26:54
    bytes = 0
    isdir = 0
    datenum = 7.3807e+05
    statinfo =

      scalar structure containing the fields:

        dev = 66306
        ino = 2.7806e+07
        mode = 33204
        modestr = -rw-rw-r--
        nlink = 1
        uid = 1001
        gid = 1001
        rdev = 0
        size = 0
        atime = 1.6021e+09
        mtime = 1.6021e+09
        ctime = 1.6021e+09
        blksize = 4096
        blocks = 0



Dmitri.
--

Dmitri A. Sergatskov <dasergatskov>
Wed 07 Oct 2020 06:10:18 AM UTC, original submission:  

In the name of God, the compassionate, the merciful.

when using "Files=dir(*.csv')" command in a folder with some files in Persian names, it produces a warning and incorrect results.
"warning: dir: 'lstat (.csv)' failed: No such file or directory
warning: called from dir at line 126 column 9"

when using "Files=dir" this behavior dose not occur.

by some try and error i found this error originates from the "glob" function. using "a=glob" return the same warning as mentioned.

for try and investigating this behavior فایل.txt can be used as a Persian file name.

thanks

Paryshaan <rouhollah>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by mmuetzel (Posted a comment)
  • -email is unavailable- added by rik5 (Posted a comment)
  • -email is unavailable- added by dasergatskov (Posted a comment)
  • -email is unavailable- added by rouhollah (Submitted the item)
  • -email is unavailable- added by rouhollah
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 7 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2020-10-10 mmuetzel StatusReady For Test Fixed
        Open/ClosedOpen Closed
    2020-10-08 mmuetzel StatusConfirmed Ready For Test
    2020-10-08 mmuetzel StatusNone Confirmed
        Release5.2.0 dev
        Summaryincorrect result of &quot;glob&quot; function handling with non-english file names Incorrect result of "dir" using wildcards with non-ASCII file names
    2020-10-07 rouhollah Carbon-Copy- Added -email is unavailable-

    Back to the top

    Powered by Savane 3.13-f8d8.
    Corresponding source code