bugGNU Octave - Bugs: bug #63524, dir doesn't do full recursive...

 
 

bug #63524: dir doesn't do full recursive search with '**'

Submitter:  Steve <alzi123>
Submitted:  Wed 14 Dec 2022 02:30:03 PM UTC
 
Category:  Octave Function Severity:  1 - Wish
Priority:  1 - Later Item Group:  Feature Request
Status:  Confirmed Assigned to:  None
Originator Name:  Open/Closed:  * Open
Release:  * dev Operating System:  * Any
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment (Rich Markup)
   

Comment Type & Canned Response:
       

 

( Jump to the original submission )

Wed 21 Dec 2022 03:13:39 PM UTC, comment #19: 

Could you please upload a diff or patch to the default branch of the Mercurial (or Git) repository of Octave?
It's hard to tell which version of the .m file you took as a base and which changes you intended.
Also, the file you uploaded has CRLF line endings. Is your editor setup correctly?

Markus Mützel <mmuetzel>
Project Member
Fri 16 Dec 2022 09:19:32 PM UTC, comment #18: 

ok. Now the dir.m is included in this bug report. Please see below.

Steve <alzi123>
Fri 16 Dec 2022 08:58:38 PM UTC, comment #17: 

please see patch #10301 for a possible functionality enhancement of the dir command.

Steve <alzi123>
Fri 16 Dec 2022 01:57:59 PM UTC, comment #16: 

yes, I retitled to be specifically about dir because the glob issue was a secondary issue at the user level while dir appeared to be an actual compatibility issue where matlab code would not execute as expected in octave.

Nicholas Jankowski <nrjank>
Project Member
Fri 16 Dec 2022 05:12:26 AM UTC, comment #15: 

IIUC, the feature request is about the support for globstar patterns with the `dir` function. That is a valid compatibility request imho.

The `dir` function happens to be implemented using the `glob` function in Octave currently. But that doesn't need to stay that way if we don't want the behavior of that function to change.

Markus Mützel <mmuetzel>
Project Member
Fri 16 Dec 2022 02:34:40 AM UTC, comment #14: 

If I understand correctly, Matlab doesn't have a 'glob' function.  The one referenced in this report is a function contributed to matlab central.  I don't think we need to worry about compatibility with random functions from that archive.  Octave's glob function was intended to provide the functionality available in the Unix function of the same name.  As far as I know, it does that job.

John W. Eaton <jwe>
Project Administrator
Thu 15 Dec 2022 10:17:01 PM UTC, comment #13: 

Changing Release to dev and OS to Any.  This is a feature addition for Matlab Compatibility.

I will say this looks like it will be modestly difficult to support.  I looked at the glob library function ("man glob") and it doesn't seem to support a "globstar" option like the shell.  The quote that implements this in bash may be under some sort of open source license which might mean we could import it into Octave.

Rik <rik5>
Project Administrator
Thu 15 Dec 2022 05:07:02 AM UTC, comment #12: 

To get recursive expansion for "**" (in bash) I need to set

$ shopt -s globstar

I was not able to do the same withing octave.

Dmitri.
--

Dmitri A. Sergatskov <dasergatskov>
Thu 15 Dec 2022 04:35:55 AM UTC, comment #11: 

And "ls" in octave behaves the same as "ls" in shell, but "dir" in octave
does not do recursive in "dir ../**" case.

Dmitri.
--

Dmitri A. Sergatskov <dasergatskov>
Thu 15 Dec 2022 04:31:42 AM UTC, comment #10: 

In bash shell it works differently. From octave build directory:

$ ls ../**
<...all octave files...>
$ ls ../**/*.m
../test/fntests.m  ../test/mk_bc_overloads_expected.m
$ ls ../*/*.m
../test/fntests.m  ../test/mk_bc_overloads_expected.m

in octave

octave:1> glob("../**")
ans =
{
  [1,1] = ../CITATION
  [2,1] = ../COPYING
  [3,1] = ../COPYRIGHT.md
  [4,1] = ../INSTALL
  [5,1] = ../Makefile.am
  [6,1] = ../Makefile.in
  [7,1] = ../README
  [8,1] = ../aclocal.m4
  [9,1] = ../autogen.sh
  [10,1] = ../autom4te.cache
  [11,1] = ../autopull.sh
  [12,1] = ../bootstrap
  [13,1] = ../bootstrap-funclib.sh
  [14,1] = ../bootstrap.conf
  [15,1] = ../build-aux
  [16,1] = ../config.in.h
  [17,1] = ../configure
  [18,1] = ../configure.ac
  [19,1] = ../doc
  [20,1] = ../etc
  [21,1] = ../examples
  [22,1] = ../gcc_opt
  [23,1] = ../gcc_tsan
  [24,1] = ../gcc_tsanO0
  [25,1] = ../gnulib
  [26,1] = ../libgnu
  [27,1] = ../libgui
  [28,1] = ../libinterp
  [29,1] = ../liboctave
  [30,1] = ../m4
  [31,1] = ../oct-conf-post-private.in.h
  [32,1] = ../oct-conf-post-public.in.h
  [33,1] = ../octave.doap
  [34,1] = ../run-octave.in
  [35,1] = ../scripts
  [36,1] = ../src
  [37,1] = ../test
}

octave:2> glob("../**/*.m")
ans =
{
  [1,1] = ../test/fntests.m
  [2,1] = ../test/mk_bc_overloads_expected.m
}

Dmitri.
--

Dmitri A. Sergatskov <dasergatskov>
Thu 15 Dec 2022 04:20:23 AM UTC, comment #9: 

Afaict, `**` as a recursive wildcard is a bash extension for glob patterns. `__wglob__` doesn't support that extension. Does it work as a recursive wildcard in glob patterns with Octave's `glob` function on Linux?

Markus Mützel <mmuetzel>
Project Member
Thu 15 Dec 2022 03:01:20 AM UTC, comment #8: 

Ok, confirming that in Matlab 2022b dir('**\*m') does recursion to full depth (seeing at least five levels deep in that working folder, so I'll assume full depth).  Octave still only does one level deep.  So that seems to be a legit compatibility request.

Regarding glob, I have no idea where that blogger got a working version of glob.  But searching Mathworks you get no function referenc, just that blog, and in Matlab 2022b with a pile of toolboxes:

>> glob('*.m')
Undefined function 'glob' for input arguments of type 'char'.

>> glob("*.m")
Unrecognized function or variable 'glob'.

>> help glob
--- glob not found. Showing help for globe instead. ---

 globe  Earth as sphere in 3-D graphics
...

>> help @string/glob

@string/glob not found.

Nicholas Jankowski <nrjank>
Project Member
Thu 15 Dec 2022 02:41:04 AM UTC, comment #7: 
Dmitri A. Sergatskov <dasergatskov>
Thu 15 Dec 2022 02:37:53 AM UTC, comment #6: 

(fixed markup)

ok.  by itself, glob is not recursive.

glob ('*.m') gives me all of the m files in the same folder.

glob ('**\*.m') appears to recurse one level deep.  so for example, in my project folder:

>> ls
octave-statistics  pkg  repelem  scatteredInterpolant  temp

all have m files, but only repelem and scatteredInterpolant have m files one level down. So:

>> glob('*.m')
ans = {}(0x0)

>> glob('**/*.m')
ans =
{
  [1,1] = repelem\repelem.m
  [2,1] = scatteredInterpolant\TriScatteredInterp.m
  [3,1] = scatteredInterpolant\delaunayn.m
  [4,1] = scatteredInterpolant\extraptest.m
  [5,1] = scatteredInterpolant\find_convex_vis_pts.m
  [6,1] = scatteredInterpolant\scatteredInterpolant.m
  [7,1] = scatteredInterpolant\testHDR.m

I'm not familiar with "filename globbing patterns .. as used by the Unix shells" so unless that pattern with glob is expected to do full recursion, this seems to be behaving okay

Nicholas Jankowski <nrjank>
Project Member
Thu 15 Dec 2022 02:36:48 AM UTC, comment #5: 

ok.  by itself, glob is not recursive.

glob ('*.m') gives me all of the m files in the same folder.

glob ('**\*.m') appears to recurse one level deep.  so for example, in my project folder:

>> ls
octave-statistics  pkg  repelem  scatteredInterpolant  temp

all have m files, but only repelem and scatteredInterpolant have m files one level down. So:

+verbamit+

>> glob('*.m')

ans = {}(0x0)

>> glob('**/*.m')

ans =
{
  [1,1] = repelem\repelem.m
  [2,1] = scatteredInterpolant\TriScatteredInterp.m
  [3,1] = scatteredInterpolant\delaunayn.m
  [4,1] = scatteredInterpolant\extraptest.m
  [5,1] = scatteredInterpolant\find_convex_vis_pts.m
  [6,1] = scatteredInterpolant\scatteredInterpolant.m
  [7,1] = scatteredInterpolant\testHDR.m


I'm not familiar with "filename globbing patterns .. as used by the Unix shells" so unless that pattern with glob is expected to do full recursion, this seems to be behaving okay

Nicholas Jankowski <nrjank>
Project Member
Thu 15 Dec 2022 02:26:44 AM UTC, comment #4: 

just to be clear, I think what you're saying is that you expect the glob function to recursively search the path below the octave source directory, and show all 6937 m-files, just as it shows the m files in your project directory.  But for the Octave program directory, it's returning empty.  similar for the dir command? (although you ran different dir commands for the two locations.)

is glob supposed to do a recursive search by default?

Nicholas Jankowski <nrjank>
Project Member
Wed 14 Dec 2022 10:25:22 PM UTC, comment #3: 

I installed Octave under Windows 10.
Explorer-search shows 6937 m-files in
'C:\Program Files\GNU Octave\Octave-7.3.0\'.
Also Everything does.

The problem is that in one directory the m files are found
but not found in the other directory.
There might be a bug.

I would expect that

glob('C:\Program Files\GNU Octave\Octave-7.3.0\**\*.m')

also shows this 6937 m-files which are physically present.

Steve <alzi123>
Wed 14 Dec 2022 03:33:18 PM UTC, comment #2: 

I'm struggling to understand which part of your examples is wrong.
Which output do you expect for which command?

Are there any .m files in 'C:\Program Files\GNU Octave\Octave-7.3.0\' (or in the immediate subfolders) for you? (There aren't any by default.)

What is the problem you'd like to solve?

Markus Mützel <mmuetzel>
Project Member
Wed 14 Dec 2022 02:50:17 PM UTC, comment #1: 

a workaround would be:

system(['dir /S /B "',p,'\*.m"']);

Steve <alzi123>
Wed 14 Dec 2022 02:30:03 PM UTC, original submission:  

Found some problem in dir / glob:

dir

cd('C:\Program Files\GNU Octave\Octave-7.3.0\');
pwd
ans = C:\Program Files\GNU Octave\Octave-7.3.0
dir('*.m')
warning: dir: nonexistent directory '*.m'
warning: called from
    dir at line 193 column 5

cd('D:\Projekte1\');
dir('**\*.m')
script001.m
script002.m
script003.m
...

glob

cd('C:\Program Files\GNU Octave\Octave-7.3.0\');
glob('**\*.m')
ans = {}(0x0)
----------------------------------------------------
cd('D:\Projekte1\');
glob('**\*.m')
ans =
{
  [1,1] = dir1\script001.m
  [2,1] = dir1\script002.m
  [3,1] = dir2\script003.m
...
}

traced it down to...

_wglob_('C:\Program Files\GNU Octave\Octave-7.3.0\**\*.m')
ans = {}(0x0)

_wglob_('D:\Projekte1\**\*.m')
ans =
{
  [1,1] = D:\Projekte1\dir1\script001.m
  [2,1] = D:\Projekte1\dir1\script002.m
  [3,1] = D:\Projekte1\dir2\script003.m
...
}

Is there a solution to this problem?

I also set all directory rights to full access for the directory
C:\Program Files\GNU Octave\Octave-7.3.0\

Steve <alzi123>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #54121:  dir.m added by alzi123 (8KiB - application/octet-stream - additional testing, enhanced code)
file #54114:  dir.m added by alzi123 (7KiB - text/plain)

 

Depends on the following items: None found

Digest:
   patch dependencies.

 

Carbon-Copy List
  • -email is unavailable- added by jwe (Posted a comment)
  • -email is unavailable- added by dasergatskov (Posted a comment)
  • -email is unavailable- added by nrjank (Posted a comment)
  • -email is unavailable- added by mmuetzel (Posted a comment)
  • -email is unavailable- added by alzi123 (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only project members can vote.

     

     

     

     

    Follow 13 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2022-12-19 rik5 Carbon-CopyRemoved 72865 -
    2022-12-17 alzi123 Attached File- Added dir.m, #54121
    2022-12-16 alzi123 Attached File- Added dir.m, #54114
    2022-12-16 nrjank Dependencies- patch #10301 is dependent
    2022-12-16 jwe Severity3 - Normal 1 - Wish
        Priority5 - Normal 1 - Later
        Item GroupMatlab Compatibility Feature Request
    2022-12-15 rik5 Release7.3.0 dev
        Operating SystemMicrosoft Windows Any
    2022-12-15 nrjank Item GroupNone Matlab Compatibility
        StatusNeed Info Confirmed
        Summarydir / glob not listing directories in some cases ! dir doesn't do full recursive search with '**'
    2022-12-14 mmuetzel StatusNone Need Info

    Back to the top

    Powered by Savane 3.10