bugfindutils - Bugs: bug #48169, find makes unnecessary syscalls

 
 

bug #48169: find makes unnecessary syscalls

Submitted by:  None
Submitted on:  Wed 08 Jun 2016 01:10:23 AM UTC  
 
Category:  find Severity:  3 - Normal
Item Group:  None Status:  None
Privacy:  Public Assigned to:  None
Originator Name:  Tavian Barnes Originator Email:  -email is unavailable-
Open/Closed:  Open Release:  4.6.0
Fixed Release:  None

Add a New Comment (Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

Sun 04 Dec 2016 05:18:13 AM UTC, comment #2: 

> Is there are measurable performance difference?


Hard to say.  find is now much better at avoiding stat() calls on non-directories, so it's overall faster.

Anyway I tracked down where some of these are coming from.  First off, the unused F_DUPFD_CLOEXEC comes from ftsfind.c doing

    curr_fd = dup_cloexec (dir_fd);

But curr_fd isn't actually used for anything any more, so it could be removed.

The duplicate stat comes from fts.c:

    /* Now read the stat info again after opening a directory to
       reveal eventual changes caused by a submount triggered by
       the traversal.  But do it only for utilities which use
       FTS_TIGHT_CYCLE_CHECK.  Therefore, only find and du
       benefit/suffer from this feature for now.  */
    LEAVE_DIR (sp, cur, "4");
    fts_stat (sp, cur, false);

so I guess that is necessary for automounts.  But maybe it's possible to avoid stat'ing before it's  opened?

fts.c could use dup_cloexec here:

    dir_fd = dup (dir_fd);
    if (0 <= dir_fd)
      set_cloexec_flag (dir_fd, true);

And finally, fts.c's opendirat() could pass O_CLOEXEC to openat() to avoid the set_cloexec_flag() call.

Tavian Barnes <tavianator>
Sun 27 Nov 2016 05:13:58 PM UTC, comment #1: 

Is there are measurable performance difference?

James Youngman <jay>
Project Administrator
Wed 08 Jun 2016 01:10:23 AM UTC, original submission:  

$ mkdir -p foo/bar/baz
$ strace find foo >/dev/null
...
newfstatat(5, "bar", {st_mode=S_IFDIR|0755, st_size=6, ...}, AT_SYMLINK_NOFOLLOW) = 0
fcntl(5, F_DUPFD_CLOEXEC, 0)            = 4
openat(5, "bar", O_RDONLY|O_NOCTTY|O_NONBLOCK|O_DIRECTORY|O_NOFOLLOW) = 6
fcntl(6, F_GETFD)                       = 0
fcntl(6, F_SETFD, FD_CLOEXEC)           = 0
fstat(6, {st_mode=S_IFDIR|0755, st_size=6, ...}) = 0
fcntl(6, F_GETFL)                       = 0x38800 (flags O_RDONLY|O_NONBLOCK|O_LARGEFILE|O_DIRECTORY|O_NOFOLLOW)
fcntl(6, F_SETFD, FD_CLOEXEC)           = 0
newfstatat(5, "bar", {st_mode=S_IFDIR|0755, st_size=6, ...}, AT_SYMLINK_NOFOLLOW) = 0
fcntl(6, F_DUPFD, 3)                    = 7
fcntl(7, F_GETFD)                       = 0
fcntl(7, F_SETFD, FD_CLOEXEC)           = 0
getdents(6, /* 3 entries */, 32768)     = 72
getdents(6, /* 0 entries */, 32768)     = 0
close(6)                                = 0
newfstatat(7, "baz", {st_mode=S_IFDIR|0755, st_size=0, ...}, AT_SYMLINK_NOFOLLOW) = 0
close(4)                                = 0
...

In particular:

- fd 4 is unused
- fcntl(6, F_SETFD, FD_CLOEXEC) happens twice, but could be totally avoided with O_CLOEXEC (I suspect the second one is from within fdopendir() though)
- newfstatat(5, "bar", AT_SYMLINK_NOFOLLOW) happens twice
- fcntl(7, F_SETFD, FD_CLOEXEC) could be avoided if fcntl(6, F_DUPFD_CLOEXEC) were used

This seems new with 4.6, at least 4.4 didn't do this.

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by tavianator (Posted a comment)
  • -email is unavailable- added by jay (Posted a comment)
  • -email is unavailable- added by None (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

     

     

    No changes have been made to this item

    Back to the top


    Powered by Savane 3.5