Thu 26 Aug 2010 07:06:32 AM UTC, comment #1:
I have attached a proof of concept patch that implements this. I'm sure it breaks some coding conventions and could use some adjustment, but it does work and passes the test suite.
Watching the action through strace, it now avoids calling stat() on excluded files at all. (It's worth noting that this optimization only comes into play when excluding/including files/dirs.)
The conditions under which this is a significant speedup appear to be fairly rare. The cost of stat()ing files seems be a low percentage of the overall runtime. Even when reading through file contents is completely avoided by doing something like --include '*.asdfasdfasdf', the speedup is only around 10% for me (searching through all of /usr recursively). Under normal conditions the speed improvement seems to be lost in noise.
However, these measurements could vary wildly between OS, filesystem, hardware, etc., for all I know. I tested the latest grep from git, on Ubuntu 10.04, on a mechanical hard drive running ext4. I also made sure to drop all disk caches before each test by echoing 3 to /proc/sys/vm/drop_caches.
I'm also confused about how the savedir code is maintained. The version in gnulib has the same interface but is significantly different from the version in grep. The gnulib version doesn't even attempt to check file type.
(file #21311)
|