bugfindutils - Bugs: bug #45930, -ignore_readdir_race ineffective...

 
 

bug #45930: -ignore_readdir_race ineffective in find 4.5.11 and 4.5.14

Submitter:  None
Submitted:  Fri 11 Sep 2015 01:05:25 PM UTC
Votes: 10
 
Category:  find Severity:  3 - Normal
Item Group:  None Status:  Fixed
Privacy:  Public Assigned to:  berny
Originator Name:  Rainer Canavan Originator Email:  -email is unavailable-
Open/Closed:  Open Release:  4.5.14
Fixed Release:  4.11.0
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Tue 17 Sep 2024 08:53:13 PM UTC, comment #8: 

Pushed with a fix for a minor typo in NEWS at:
https://git.sv.gnu.org/cgit/findutils.git/commit/?id=889d001ab75

Bernhard Voelker <berny>
Group administrator
Mon 16 Sep 2024 10:13:18 PM UTC, comment #7: 

I propose to push the attached, as this make the race already
much more unlikely.  This is included in downstream Redhat and
Fedora since a while.

(file #56434)

Bernhard Voelker <berny>
Group administrator
Wed 04 Sep 2019 08:42:12 PM UTC, comment #6: 

No, it's not fixed.  I've identified at least one place where catching ENOENT helps ...


diff --git a/find/ftsfind.c b/find/ftsfind.c
index 0d96c4ca..6aeac28c 100644
--- a/find/ftsfind.c
+++ b/find/ftsfind.c
@@ -401,6 +401,10 @@ consider_visiting (FTS *p, FTSENT *ent)
            }
          else
            {
+             /* Ignore unlink() error for vanished files.  */
+             if (ENOENT == ent->fts_errno && options.ignore_readdir_race)
+                 return;
+
              nonfatal_target_file_error (ent->fts_errno, ent->fts_path);
              /* Continue despite the error, as file name without stat info
               * might be better than not even processing the file name. This


..., but there seem to be more.
Here, the reproducer in #4 still "works".

Bernhard Voelker <berny>
Group administrator
Sun 01 Sep 2019 08:58:38 AM UTC, comment #5: 

Trying the reproducer in #4 with Findutils 4.7.0, I see no problems.    Bernhard, did you fix this already?   If the problem is partially-fixed, is it fixed enough that we can close the bug?

James Youngman <jay>
Group administrator
Thu 13 Jun 2019 06:18:39 AM UTC, comment #4: 

The issue can be triggered with a shorter command line.

Run this in one terminal:

while env true; do sleep .001; done

and in another one this find(1) command:

$ find /proc -ignore_readdir_race -maxdepth 3 > /dev/null


With that, find(1) is hitting the error pretty reliably.

FWIW: Interestingly, 'oldfind', i.e., the old non-FTS implementation even runs into an endless loop:

...
/home/berny/findutils/find/oldfind: ‘/proc/1143/net’: Invalid argument
/home/berny/findutils/find/oldfind: ‘/proc/1143/net’: Invalid argument
/home/berny/findutils/find/oldfind: ‘/proc/1143/net’: Invalid argument
/home/berny/findutils/find/oldfind: ‘/proc/1143/net’: Invalid a^C


Bernhard Voelker <berny>
Group administrator
Thu 06 Jun 2019 09:04:54 AM UTC, comment #3: 

Hello,

I can confirm "-ignore_readdir_race" not working:
- On CentOS-7 with find v4.5.11
- On OpenSuse with find v4.5.12

EXAMPLE:
root|dev-opensuse-server01|10:57|0|~: find /proc -ignore_readdir_race -maxdepth 3 -path "/proc/[0-9]*/fd/*" -type l \( -lname '/dev/shm/*' -prune -o -lname '*(deleted)' -printf '"%p (absent) (%Y)"\n' -o -printf '"%p (present) (%Y)"\n' \) > /dev/null; echo -e "RETVALUE: ${?}"
find: ‘/proc/4063’: No such file or directory
find: ‘/proc/4064’: No such file or directory
find: ‘/proc/4069’: No such file or directory
find: ‘/proc/4072’: No such file or directory
RETVALUE: 1

EXPECTED RESULT:
- No error about missing directories/files
- Return value of zero (0)

Either it is broken or I misunderstood the purpose of this feature...

Anonymous
Fri 18 Aug 2017 01:20:42 AM UTC, comment #2: 

I can confirm the issue with CentOS7. Package info: findutils-4.5.11-5.el7.x86_64

Have a CRON that runs on multiple machine at the same time. Part of the script remove old log file in a common NFS folder.
It uses 'find' for this and checks the file timestamp (mtime)

It's pretty much guaranteed to get into a race condition with  this specific setup (NFS mount..).

Using -ignore_readdir_race does not seem to have any effect.


Alex <arekkusu>
Fri 11 Sep 2015 01:17:57 PM UTC, comment #1: 

I haven't tried other filesystems or kernels, the ones used are xfs (rw,noatime,attr2,inode64,noquota) and linux 3.10.0-229.11.1.el7.x86_64

Rainer Canavan <canavan>
Fri 11 Sep 2015 01:05:25 PM UTC, original submission:  

We have a cron job that is used to clean cache directories. There's a log going on in that directory, especially files getting moved from temporary names to their final names after they have been written completely. We have noticed that the find that is supplied with Centos7 (version 4.5.11) as well as a find 4.5.14 built from unmodified source frequently prints "No such file or directory" errors for the initial filenames of the temporary files, despite the fact that we have specified -ignore_readdir_race, e.g.:

/usr/bin/find: ‘./engine/projects/portal/istmp1NYV9b’: No such file or directory

This does not happen with a find 4.4.2 compiled from source. A trivial find in the "busy" directory is sufficient to triffer the error message almost 100% of the time:

find . -ignore_readdir_race > /dev/null


Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #56434:  0001-find-ignore-more-vanished-entries-with-ignore_readdi.patch added by berny (2KiB - text/x-patch - [PATCH] find: ignore more vanished entries with -ignore_readdir_race)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by jay (Posted a comment)
  • -email is unavailable- added by berny (Posted a comment)
  • -email is unavailable- added by arekkusu (Posted a comment)
  • -email is unavailable- added by arekkusu (Voted in favor of this item)
  • -email is unavailable- added by canavan (Posted a comment)
  • -email is unavailable- added by None (Submitted the item)
  •  

    There are 10 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 6 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2024-09-17 berny StatusCode Review Fixed
    2024-09-16 berny Attached File- Added 0001-find-ignore-more-vanished-entries-with-ignore_readdi.patch, #56434
        StatusNone Code Review
        Fixed ReleaseNone 4.11.0
    2019-09-01 jay Assigned toNone berny
    2017-08-18 arekkusu Carbon-Copy- Added arekkusu

    Back to the top

    Powered by Savane 3.13-ec04.
    Corresponding source code