patchfindutils - Patches: patch #9705, make ftsfind honour the...

 
 

You are not allowed to post comments on this tracker with your current authentication level.

patch #9705: make ftsfind honour the -ignore_readdir_race option

Submitter:  Gajendran Kanapathipillai <gajendrank>
Submitted:  Mon 15 Oct 2018 03:25:13 PM UTC
Votes: 10
 
Category:  find bugfix Priority:  5 - Normal
Status:  None Privacy:  Public
Assigned to:  berny Originator Name: 
Open/Closed:  Open Release:  None
Fixed Release:  None

Mon 15 Oct 2018 10:00:12 PM UTC, comment #5: 

I couldn't reproduce with that test case on XFS, but I played a bit
with trying to create races.

It seems there are more race conditions which -ignore_readdir_race
should catch.  The following creates 'tmp/d' alternatively as a
directory and a regular file:

  $ while mkdir -p tmp tmp/d && rm -fr tmp/d && touch tmp/d && rm -f tmp/d ; do : ; done

Now, in a second terminal, the following command should ideally run without errors:

  $ while : ; do ~/findutils/find/find -D search -ignore_readdir_race -delete || break; done

It turns out, that find terminates at least when it gets an error
from newstatat(), and from open().
Needs some more investigation ...


Bernhard Voelker <berny>
Group administrator
Mon 15 Oct 2018 07:54:40 PM UTC, comment #4: 

That was my finding too.  This only happens on certain installations and not others.  On a system where a physical parition (e.g. /dev/sda2) mounted with xfs had the same output as you've shown below.

However, with the following system (CentOS 7), it produces FTS_NS instead of FTS_NSOK and gets into this failure.

> uname -a

Linux 4.4.109-1.el7.elrepo.x86_64 #1 SMP Tue Jan 2 17:24:53 EST 2018 x86_64 x86_64 x86_64 GNU/Linux

> mount | grep " / "

/dev/mapper/vg_root-lv_root on / type xfs (rw,relatime,seclabel,attr2,inode32,noquota)

This FTS_NS vs FTS_NSOK condition appears to be related to filesystem/mount options only because I was able to get both FTS_NS and FTS_NSOK on the same machine (using different mount points of course).


Gajendran Kanapathipillai <gajendrank>
Mon 15 Oct 2018 06:39:48 PM UTC, comment #3: 

[re-opeing the issue]

Hmm, I don't get this error, and the '-D search' output looks different for me,
and I ran it many times:

consider_visiting (early): '.': fts_info=FTS_D , fts_level= 0, prev_depth=-2147483648 fts_path='.', fts_accpath='.'
consider_visiting (late): '.': fts_info=FTS_D , isdir=1 ignore=1 have_stat=1 have_type=1
consider_visiting (early): './5': fts_info=FTS_NSOK, fts_level= 1, prev_depth=0 fts_path='./5', fts_accpath='5'
consider_visiting (late): './5': fts_info=FTS_NSOK, isdir=0 ignore=0 have_stat=0 have_type=1
find deleted: ./5
consider_visiting (early): './7': fts_info=FTS_NSOK, fts_level= 1, prev_depth=1 fts_path='./7', fts_accpath='7'
consider_visiting (late): './7': fts_info=FTS_NSOK, isdir=0 ignore=0 have_stat=0 have_type=1
find deleted: ./7
consider_visiting (early): './8': fts_info=FTS_NSOK, fts_level= 1, prev_depth=1 fts_path='./8', fts_accpath='8'
consider_visiting (late): './8': fts_info=FTS_NSOK, isdir=0 ignore=0 have_stat=0 have_type=1
removed '1'
removed '2'
removed '3'
removed '4'
removed '6'
removed '9'
removed '10'
find deleted: ./8
consider_visiting (early): './9': fts_info=FTS_NSOK, fts_level= 1, prev_depth=1 fts_path='./9', fts_accpath='9'
consider_visiting (late): './9': fts_info=FTS_NSOK, isdir=0 ignore=0 have_stat=0 have_type=1
find deleted: ./9
consider_visiting (early): './2': fts_info=FTS_NSOK, fts_level= 1, prev_depth=1 fts_path='./2', fts_accpath='2'
consider_visiting (late): './2': fts_info=FTS_NSOK, isdir=0 ignore=0 have_stat=0 have_type=1
find deleted: ./2
consider_visiting (early): './10': fts_info=FTS_NSOK, fts_level= 1, prev_depth=1 fts_path='./10', fts_accpath='10'
consider_visiting (late): './10': fts_info=FTS_NSOK, isdir=0 ignore=0 have_stat=0 have_type=1
find deleted: ./10
consider_visiting (early): './1': fts_info=FTS_NSOK, fts_level= 1, prev_depth=1 fts_path='./1', fts_accpath='1'
consider_visiting (late): './1': fts_info=FTS_NSOK, isdir=0 ignore=0 have_stat=0 have_type=1
find deleted: ./1
consider_visiting (early): './3': fts_info=FTS_NSOK, fts_level= 1, prev_depth=1 fts_path='./3', fts_accpath='3'
consider_visiting (late): './3': fts_info=FTS_NSOK, isdir=0 ignore=0 have_stat=0 have_type=1
find deleted: ./3
consider_visiting (early): './6': fts_info=FTS_NSOK, fts_level= 1, prev_depth=1 fts_path='./6', fts_accpath='6'
consider_visiting (late): './6': fts_info=FTS_NSOK, isdir=0 ignore=0 have_stat=0 have_type=1
find deleted: ./6
consider_visiting (early): './4': fts_info=FTS_NSOK, fts_level= 1, prev_depth=1 fts_path='./4', fts_accpath='4'
consider_visiting (late): './4': fts_info=FTS_NSOK, isdir=0 ignore=0 have_stat=0 have_type=1
find deleted: ./4
consider_visiting (early): '.': fts_info=FTS_DP, fts_level= 0, prev_depth=1 fts_path='.', fts_accpath='.'
consider_visiting (late): '.': fts_info=FTS_DP, isdir=1 ignore=0 have_stat=1 have_type=1
find exit status: 0
find time: 55.07

I'm on ext4.  Do you have a special environment? Or a tweaked reproducer?
Thanks.

Bernhard Voelker <berny>
Group administrator
Mon 15 Oct 2018 04:58:27 PM UTC, comment #2: 

Actually, with the ftsfind, the test never gets to the code where earlier patch (0afb2efad) is fixing the issue.  Instead, the ftsfind errors early and sets the errno to "1".  Which causes the "-ignore_readdir_race -delete" combination to still fail and return error.

The patch attached prevents the ftsfind to error early and and set the errno to 1, so the original patch (0afb2efad) is reached and handled correctly.

Here's the test loop output without the attahed patch:

consider_visiting (early): ‘./6’: fts_info=FTS_F , fts_level= 1, prev_depth=1 fts_path=‘./6’, fts_accpath=‘6’
consider_visiting (late): ‘./6’: fts_info=FTS_F , isdir=0 ignore=0 have_stat=1 have_type=1
removed ‘7’
removed ‘8’
removed ‘9’
removed ‘10’
find deleted: ./6
consider_visiting (early): ‘./7’: fts_info=FTS_NS, fts_level= 1, prev_depth=1 fts_path=‘./7’, fts_accpath=‘7’
/bin/find: ‘./7’: No such file or directory
consider_visiting (late): ‘./7’: fts_info=FTS_NS, isdir=0 ignore=0 have_stat=0 have_type=0



Gajendran Kanapathipillai <gajendrank>
Mon 15 Oct 2018 03:44:39 PM UTC, comment #1: 

That patch has already been applied 2018-03-14:
  https://git.sv.gnu.org/cgit/findutils.git/commit/?id=0afb2efad

Therefore, I'm marking this patch as done.

 Bernhard Voelker <berny>
Group administrator
Mon 15 Oct 2018 03:25:13 PM UTC, original submission:  

In the ftsfind, the ignore_readdir_race is not considered when the file goes missing.  Please refer to the following fix for the issue and how this patch was tested:


commit 0afb2efada7e435ae18ef7d3db0758464189f44f
Author: Bernhard Voelker <mail@bernhard-voelker.de>
Date:   Tue Jan 30 23:30:09 2018 +0100

    find: make -delete honour the -ignore_readdir_race option
   
    * find/pred.c (pred_delete): Return true when the -ignore_readdir_race
    option is active and unlinkat() came back with ENOENT.
    * doc/find.texi (Option -ignore_readdir_race): Document the change.
    (Action -delete): Likewise.
    * find/find.1: Likewise.
    * NEWS (Bug Fixes): Mention the fix.
   
    For now, it seems a bit hard to add a proper test for this,
    so the following shell snippet demonstrates the race:
   
      $ seq 10 | xargs touch
      $ env time -f 'find exit status: %x\nfind time: %e' \
          find -ignore_readdir_race -type f \
            -delete \
            -exec sh -c 'sleep $(basename {})' \; \
            -printf 'find deleted: %p\n' \
            & \
        sleep 20; \
        seq 10 | xargs rm -fv; \
        wait $!
   
    Reported by Alexander Golubev in
    https://savannah.gnu.org/bugs/?52981




Gajendran Kanapathipillai <gajendrank>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attached Files
file #45201:  0001-find-make-ftsfind-honour-the-ignore_readdir_race-opt.patch added by gajendrank (791B - text/plain)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by antigoon2 (Voted in favor of this item)
  • -email is unavailable- added by berny (Posted a comment)
  • -email is unavailable- added by jansmets
  • -email is unavailable- added by gajendrank (Submitted the item)

    •  

    There are 10 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

     

    Follow 8 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2018-10-22 antigoon2 Carbon-Copy- Added antigoon2
    2018-10-15 berny StatusDone None
    2018-10-15 berny Open/ClosedClosed Open
    2018-10-15 berny StatusNone Done
        Assigned toNone berny
        Open/ClosedOpen Closed
    2018-10-15 jansmets Carbon-Copy- Added jansmets
    2018-10-15 gajendrank Attached File- Added 0001-find-make-ftsfind-honour-the-ignore_readdir_race-opt.patch, #45201

    Back to the top

    Copyright © 2024 Free Software Foundation, Inc.
    Verbatim copying and distribution of this entire article is permitted in any medium, provided this notice is preserved.
    The Levitating, Meditating, Flute-playing Gnu logo is a GNU GPL'ed image provided by the Nevrax Design Team.
    Page source code

    Powered by Savane 3.14-8eb0.
    Corresponding source code