bugfindutils - Bugs: bug #64171, find is missing folders on my mac

 
 

bug #64171: find is missing folders on my mac

Submitter:  Rick Kilgore <rkilgore>
Submitted:  Tue 09 May 2023 04:39:13 PM UTC
   
 
Category:  find Severity:  3 - Normal
Item Group:  Wrong result Status:  None
Privacy:  Public Assigned to:  None
Originator Name:  Rick Kilgore Open/Closed:  Open
Release:  4.9.0 Fixed Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Tue 16 May 2023 01:07:53 AM UTC, comment #8: 

I also just tried with the following ./configure options and it didn't help.  Still got the same results.


./configure --disable-leaf-optimisation --disable-d_type-optimization --disable-largefile --disable-year2038 --disable-threads --without-selinux --disable-cross-guesses


 Rick Kilgore <rkilgore>
Tue 16 May 2023 01:06:16 AM UTC, comment #7: 

Seems like my last comment got lost.

I do see the same behavior if I build it myself.  If I run with -D all and without any -maxdepth arg (the scenario that fails), the ./opt folder does not appear anywhere in the output.

If I run with -D all and also with -maxdepth 2, here is the relevant bit from the output concerning the ./opt folder:

consider_visiting (early): ‘./opt’: fts_info=FTS_D , fts_level= 1, prev_depth=1 fts_path=‘./opt’, fts_accpath=‘opt’
consider_visiting (late): ‘./opt’: fts_info=FTS_D , isdir=1 ignore=0 have_stat=1 have_type=1
consider_visiting (early): ‘./opt/elasticsearch-7.17.9’: fts_info=FTS_D , fts_level= 2, prev_depth=1 fts_path=‘./opt/elasticsearch-7.17.9’, fts_accpath=‘elasticsearch-7.17.9’
consider_visiting (late): ‘./opt/elasticsearch-7.17.9’: fts_info=FTS_D , isdir=1 ignore=0 have_stat=1 have_type=1
consider_visiting (early): ‘./opt/elasticsearch-7.17.9’: fts_info=FTS_DP, fts_level= 2, prev_depth=2 fts_path=‘./opt/elasticsearch-7.17.9’, fts_accpath=‘elasticsearch-7.17.9’
consider_visiting (late): ‘./opt/elasticsearch-7.17.9’: fts_info=FTS_DP, isdir=1 ignore=1 have_stat=1 have_type=1
consider_visiting (early): ‘./opt/.DS_Store’: fts_info=FTS_NSOK, fts_level= 2, prev_depth=2 fts_path=‘./opt/.DS_Store’, fts_accpath=‘.DS_Store’
consider_visiting (late): ‘./opt/.DS_Store’: fts_info=FTS_NSOK, isdir=0 ignore=0 have_stat=0 have_type=1
consider_visiting (early): ‘./opt/homebrew’: fts_info=FTS_D , fts_level= 2, prev_depth=2 fts_path=‘./opt/homebrew’, fts_accpath=‘homebrew’
consider_visiting (late): ‘./opt/homebrew’: fts_info=FTS_D , isdir=1 ignore=0 have_stat=1 have_type=1
consider_visiting (early): ‘./opt/homebrew’: fts_info=FTS_DP, fts_level= 2, prev_depth=2 fts_path=‘./opt/homebrew’, fts_accpath=‘homebrew’
consider_visiting (late): ‘./opt/homebrew’: fts_info=FTS_DP, isdir=1 ignore=1 have_stat=1 have_type=1
consider_visiting (early): ‘./opt/local’: fts_info=FTS_D , fts_level= 2, prev_depth=2 fts_path=‘./opt/local’, fts_accpath=‘local’
consider_visiting (late): ‘./opt/local’: fts_info=FTS_D , isdir=1 ignore=0 have_stat=1 have_type=1
consider_visiting (early): ‘./opt/local’: fts_info=FTS_DP, fts_level= 2, prev_depth=2 fts_path=‘./opt/local’, fts_accpath=‘local’
consider_visiting (late): ‘./opt/local’: fts_info=FTS_DP, isdir=1 ignore=1 have_stat=1 have_type=1
consider_visiting (early): ‘./opt/X11’: fts_info=FTS_D , fts_level= 2, prev_depth=2 fts_path=‘./opt/X11’, fts_accpath=‘X11’
consider_visiting (late): ‘./opt/X11’: fts_info=FTS_D , isdir=1 ignore=0 have_stat=1 have_type=1
consider_visiting (early): ‘./opt/X11’: fts_info=FTS_DP, fts_level= 2, prev_depth=2 fts_path=‘./opt/X11’, fts_accpath=‘X11’
consider_visiting (late): ‘./opt/X11’: fts_info=FTS_DP, isdir=1 ignore=1 have_stat=1 have_type=1
consider_visiting (early): ‘./opt’: fts_info=FTS_DP, fts_level= 1, prev_depth=2 fts_path=‘./opt’, fts_accpath=‘opt’
consider_visiting (late): ‘./opt’: fts_info=FTS_DP, isdir=1 ignore=1 have_stat=1 have_type=1


 Rick Kilgore <rkilgore>
Wed 10 May 2023 04:50:38 PM UTC, comment #6: 


> Rick, can you reproduce this issue with findutils built from source code?


I'm pretty sure I tried that and the answer is yes I can reproduce it.  But I will double check this afternoon.

I will also gather the log without -maxdepth, but I think I also did that last night and the difference was that the /opt folder did not even appear in the output without -maxdepth.  It skipped it and didn't even log anything about it.  Again, I will double check this afternoon.

 Rick Kilgore <rkilgore>
Wed 10 May 2023 11:21:24 AM UTC, comment #5: 

Rick, can you reproduce this issue with findutils built from source code?   If not, then the place to report this bug is whoever packaged the program you are using.  But if this is reproducible on a binary you built from http://ftp.gnu.org/pub/gnu/findutils/findutils-4.9.0.tar.xz then we should investigate further.

If you can reproduce this with a from-source build, please try a run with "find -D all" with no -maxdepth.   Please compare the debug logs (with and without the depth limit, that is, with and without the problem you are reporting) to find out the difference in how /opt is being handled.

Please don't attach the (full) debug log here, since doubtless at least some of the names of the files on your computer are private.

 James Youngman <jay>
Group administrator
Wed 10 May 2023 07:16:55 AM UTC, comment #4: 

comment #2:

> What do you see if you turn on debug output?  For example,
> ...


I attached the output of "sudo gfind -D all / -maxdepth 2" as debug.log.  Remember that this command actually works.  It works fine at least as far as -maxdepth 8, but if I don't supply a -maxdepth argument at all, then it fails.

Here are the other answers:

[txbook /]: ( cd / && gls -a / | while read i ; do [ -d "$i" ] && printf x; done ) | wc -c
      19
[txbook /]: gls -a1F | grep -c '/$'
15
[txbook /]: gstat /
  File: /
  Size: 640             Blocks: 0          IO Block: 4096   directory
Device: 1,14    Inode: 2           Links: 20
Access: (0755/drwxr-xr-x)  Uid: (    0/    root)   Gid: (    0/   wheel)
Access: 2023-04-01 09:46:50.000000000 -0700
Modify: 2023-04-01 09:46:50.000000000 -0700
Change: 2023-04-01 09:46:50.000000000 -0700
 Birth: 2023-04-01 09:46:50.000000000 -0700


Regarding the link counts, there are 4 soft links to directories, but none of them are /opt. (.VolumeIcon.icns is a broken link).

[txbook /]: gls -a1F
./
../
.VolumeIcon.icns@
.file
.vol/
Applications/
Library/
System/
Users/
Volumes/
bin/
cores/
dev/
etc@
home@
opt/
private/
sbin/
tmp@
usr/
var@


(file #54716)

 Rick Kilgore <rkilgore>
Tue 09 May 2023 11:30:10 PM UTC, comment #3: 

Just a data point. I can't reproduce this (using gfind from macports and macos-10.14.6). My /opt and /Users appear in the output. I could understand some parts of /Users not appearing in recent versions of macOS unless gfind (or maybe Terminal.app) has Full Disk Access, but that wouldn't explain the complete absence of /Users or /opt.

 raf <raf>
Tue 09 May 2023 08:15:09 PM UTC, comment #2: 

What do you see if you turn on debug output?  For example,


find -D all / -maxdepth 2


How many direct children (including . and ..) does / have?   What is its hard link count?

e.g. on my system the answer is 32 for both:


jupiter:/$ ( cd / && ls -a / | while read i ; do [ -d "$i" ] && printf x; done ) | wc -c
32
jupiter:/$ ls -a1F | grep -c '/$'
32
jupiter:/$ stat /
  File: /
  Size: 40                Blocks: 18         IO Block: 2560   directory
Device: 18h/24d        Inode: 34          Links: 32
Access: (0755/drwxr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2023-05-09 21:11:05.083727547 +0100
Modify: 2023-05-01 08:34:01.653993646 +0100
Change: 2023-05-01 08:34:01.653993646 +0100
 Birth: -




 James Youngman <jay>
Group administrator
Tue 09 May 2023 04:44:03 PM UTC, comment #1: 

I forgot to mention that it finds everything in /opt (afaict) if I do the search directly on /opt.

> cd /
> sudo gfind opt > ~/files.txt

I see all the files I expect to in ~/files.txt.

 Rick Kilgore <rkilgore>
Tue 09 May 2023 04:39:13 PM UTC, original submission:  

Repro steps (I hope it happens not just on my machine):


> cd /
> sudo gfind . > files.txt


expect:

all files under / (except maybe some special / dev files?) should be listed in files.txt

actual:

there is a folder directly under / named /opt.  That whole folder is missing.  Others are as well, including /Users.  I don't know what all is missing.

If I limit the depth of the search, it finds all the missing folders I have tested.  Here is the output of my gdf command, in case it helps:

Filesystem            1K-blocks     Used  Available Use% Mounted on
/dev/disk3s3s1       1948455240 90915268 1857539972   5% /
/dev/disk3s6         1948455240 90915268 1857539972   5% /System/Volumes/VM
/dev/disk3s4         1948455240 90915268 1857539972   5% /System/Volumes/Preboot
/dev/disk3s2         1948455240 90915268 1857539972   5% /System/Volumes/Update
/dev/disk1s2             512000    20376     491624   4% /System/Volumes/xarts
/dev/disk1s1             512000    20376     491624   4% /System/Volumes/iSCPreboot
/dev/disk1s3             512000    20376     491624   4% /System/Volumes/Hardware
Cryptomator@macfuse0 1948455240 90915268 1857539972   5% /Users/rkilgore/crypto
/dev/disk3s3         1948455240 90915268 1857539972   5% /System/Volumes/Update/mnt1


 Rick Kilgore <rkilgore>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #54716:  debug.log added by rkilgore (207KiB - application/octet-stream)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by raf (Posted a comment)
  • -email is unavailable- added by jay (Posted a comment)
  • -email is unavailable- added by rkilgore (Submitted the item)

    •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follows 1 latest change.

    Date Changed by Updated Field Previous Value => Replaced by
    2023-05-10 rkilgore Attached File- Added debug.log, #54716

    Back to the top

    Copyright © 2024 Free Software Foundation, Inc.
    Verbatim copying and distribution of this entire article is permitted in any medium, provided this notice is preserved.
    The Levitating, Meditating, Flute-playing Gnu logo is a GNU GPL'ed image provided by the Nevrax Design Team.
    Page source code

    Powered by Savane 3.14-f13d.
    Corresponding source code