bugfindutils - Bugs: bug #48169, find makes unnecessary syscalls

 
 

bug #48169: find makes unnecessary syscalls

Submitter:  None
Submitted:  Wed 08 Jun 2016 01:10:23 AM UTC
   
 
Category:  find Severity:  3 - Normal
Item Group:  None Status:  Postponed
Privacy:  Public Assigned to:  jay
Originator Name:  Tavian Barnes Originator Email:  -email is unavailable-
Open/Closed:  Closed Release:  4.6.0
Fixed Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Sun 01 Sep 2019 08:00:10 AM UTC, comment #3: 

We have quite a few open bugs and in the absence of a (reported) measurable performance difference, I'm going to close this bug.

I'm not saying the bug report is incorrect, just that in the absence of a measurable perfoemance impact it doesn't seem worthwhile to work on this bug (and in so doing, neglect others).

Please re-open this bug or create a new one if there is, in fact, a measurable performance difference.

James Youngman <jay>
Group administrator
Sun 04 Dec 2016 05:18:13 AM UTC, comment #2: 


> Is there are measurable performance difference?


Hard to say.  find is now much better at avoiding stat() calls on non-directories, so it's overall faster.

Anyway I tracked down where some of these are coming from.  First off, the unused F_DUPFD_CLOEXEC comes from ftsfind.c doing

    curr_fd = dup_cloexec (dir_fd);

But curr_fd isn't actually used for anything any more, so it could be removed.

The duplicate stat comes from fts.c:

    /* Now read the stat info again after opening a directory to
       reveal eventual changes caused by a submount triggered by
       the traversal.  But do it only for utilities which use
       FTS_TIGHT_CYCLE_CHECK.  Therefore, only find and du
       benefit/suffer from this feature for now.  */
    LEAVE_DIR (sp, cur, "4");
    fts_stat (sp, cur, false);

so I guess that is necessary for automounts.  But maybe it's possible to avoid stat'ing before it's  opened?

fts.c could use dup_cloexec here:

    dir_fd = dup (dir_fd);
    if (0 <= dir_fd)
      set_cloexec_flag (dir_fd, true);

And finally, fts.c's opendirat() could pass O_CLOEXEC to openat() to avoid the set_cloexec_flag() call.

Tavian Barnes <tavianator>
Sun 27 Nov 2016 05:13:58 PM UTC, comment #1: 

Is there are measurable performance difference?

James Youngman <jay>
Group administrator
Wed 08 Jun 2016 01:10:23 AM UTC, original submission:  

$ mkdir -p foo/bar/baz
$ strace find foo >/dev/null
...
newfstatat(5, "bar", {st_mode=S_IFDIR|0755, st_size=6, ...}, AT_SYMLINK_NOFOLLOW) = 0
fcntl(5, F_DUPFD_CLOEXEC, 0)            = 4
openat(5, "bar", O_RDONLY|O_NOCTTY|O_NONBLOCK|O_DIRECTORY|O_NOFOLLOW) = 6
fcntl(6, F_GETFD)                       = 0
fcntl(6, F_SETFD, FD_CLOEXEC)           = 0
fstat(6, {st_mode=S_IFDIR|0755, st_size=6, ...}) = 0
fcntl(6, F_GETFL)                       = 0x38800 (flags O_RDONLY|O_NONBLOCK|O_LARGEFILE|O_DIRECTORY|O_NOFOLLOW)
fcntl(6, F_SETFD, FD_CLOEXEC)           = 0
newfstatat(5, "bar", {st_mode=S_IFDIR|0755, st_size=6, ...}, AT_SYMLINK_NOFOLLOW) = 0
fcntl(6, F_DUPFD, 3)                    = 7
fcntl(7, F_GETFD)                       = 0
fcntl(7, F_SETFD, FD_CLOEXEC)           = 0
getdents(6, /* 3 entries */, 32768)     = 72
getdents(6, /* 0 entries */, 32768)     = 0
close(6)                                = 0
newfstatat(7, "baz", {st_mode=S_IFDIR|0755, st_size=0, ...}, AT_SYMLINK_NOFOLLOW) = 0
close(4)                                = 0
...

In particular:

- fd 4 is unused
- fcntl(6, F_SETFD, FD_CLOEXEC) happens twice, but could be totally avoided with O_CLOEXEC (I suspect the second one is from within fdopendir() though)
- newfstatat(5, "bar", AT_SYMLINK_NOFOLLOW) happens twice
- fcntl(7, F_SETFD, FD_CLOEXEC) could be avoided if fcntl(6, F_DUPFD_CLOEXEC) were used

This seems new with 4.6, at least 4.4 didn't do this.

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by tavianator (Posted a comment)
  • -email is unavailable- added by jay (Posted a comment)
  • -email is unavailable- added by None (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 3 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2019-09-01 jay StatusNone Postponed
        Assigned toNone jay
        Open/ClosedOpen Closed

    Back to the top

    Powered by Savane 3.13-f8d8.
    Corresponding source code