Sat 14 Jul 2007 12:58:47 PM UTC, comment #7:
As to when to use the cache vs. avoid it, I actually meant the opposite from what you wrote. Maybe it's more clear with a practical example:
$ time make -C fltk-1.1.7, up-to-date, on a (slow) Ubuntu box:
make (3.81b4, as is):
- 3.8 sec
make (cvs, no default_terminal_rules, no cache):
- 2.0 sec
|
Thu 12 Jul 2007 07:57:48 AM UTC, comment #6:
I haven't looked into it carefully, but it's not immediately clear to me that it's a simple thing to avoid the directory cache for chained/intermediate rules, vs. any other kind of rule.
The disadvantage with the "is_stale" boolean is that every time something goes stale you have to walk through the entire cache and reset all the boolean values.
If you use a counter then you don't have to touch anything in the cache until and unless you need to look at that cached data, then you compare (and update) the counter kept with that set of data.
It may be that our caches are not large enough to make a difference, practically speaking; I'm not sure.
|
Wed 11 Jul 2007 06:15:46 PM UTC, comment #5:
Timestamps are a different matter, gmake doesn't use the cache to check timestamps of files. Also, what I meant was chained rules with intermediate files, not first-level implicit rules. Basically it is so that gmakes spends all win from the cache on it's own built-in chained and match-anything rules. For the user the cache brings no benefit, only the known inconsistencies.
However it is maybe right that two different behaviours is too confusing. Probably it would be better to use no cache at all and get rid of some exotic built-in rules instead.
As to empty the cache after each subprocess, well it could work too. I don't quite see the deal with the counter though, shouldn't a boolean 'is_stale' suffice?
|
Tue 10 Jul 2007 06:50:59 AM UTC, comment #4:
Something like that could be tricky to accomplish, since make is highly recursive: when make starts checking for timestamps on files it doesn't know whether that file was found using implicit or explicit rules, etc.
Plus, I think it would be even more confusing to have some kinds of targets (implicit) behave unexpectedly while others (explicit) behaved as expected, but more slowly.
To fix this problem we need to add a way to determine if the cache is stale.
My idea on how to do this is for make to keep a counter that is incremented every time make invokes a subprocess. Or maybe, from a correctness point of view, it should be updated every time a subprocess finishes. Then when a directory was cached it would store the current value of that counter. The next time the cached data was consulted, if the current counter was greater than the stored counter the cache would be considered stale and new data would be read.
This would significantly reduce the efficiency of the cache, but it would increase correctness and the cache would still be useful while make is looking for things to build--for situations where there's very little or nothing to do the cache would still be a big win.
|
Mon 09 Jul 2007 08:25:24 PM UTC, comment #3:
It would maybe make sense to rely on the directory cache only for intermediate search and match-anything rules. Because these two alone increase the number of file-queries by a factor of about 10. That is just from the 5 built-in rules (which moreover likely nobody ever uses).
|
Sat 07 Jul 2007 09:01:14 PM UTC, comment #2:
It's not exactly correct to say that GNU make caches directories from the 10th on, but you're on exactly the right track; thanks for the note.
What make actually does is cache EVERY directory... BUT it caches them "lazily", AND it only allows 10 directories to be open at any one time. Basically, when make wants to check a file in a directory it opens the directory for reading and looks for that file. As it reads the directory it caches the contents. Once the file is found, it leaves the directory open but stops reading. If make looks for another file in that directory and it's not in the cache, make continues reading (and caching) where it left off. If it gets to the end of the directory it closes the directory.
In order to avoid too many directories being open by make at one time, if make gets to a point where more than 10 directories are open, it will completely read and cache the current (latest) directory, then close it.
The issue of make's caching is well-known (see bug #443 for example). In your case it takes an odd twist but this is definitely the issue. A completely correctly-written makefile shouldn't run across cache problems, but I recognize there are situations where it's not possible to write that makefile. However, in your case I think you can do what you want. Change your makefile like this:
This has some other advantages over the method you were using (for one thing, your example does not work properly if any of the sub-makes fail).
|
Sat 07 Jul 2007 06:53:28 PM UTC, comment #1:
gmake caches directories from the 10th on. So it will not see a file in those directories if it didn't exist initially (except under windows NT, where it checks the directory timestamps).
Just looks like foo1..8 plus lib, include and . were one more than 10.
|
Sat 07 Jul 2007 08:09:38 AM UTC, original submission:
With GNU make v3.81 (as well as earlier versions, eg. 3.81beta4 and v3.80), the vpath directive for libraries seems to "forget" entries under certain circumstances.
Example:
A project with the following directory layout:
Makefile
foo1/
foo1/Makefile
foo2/
foo2/Makefile
foo3/
foo3/Makefile
foo4/
foo4/Makefile
foo5/
foo5/Makefile
foo6/
foo6/Makefile
foo7/
foo7/Makefile
foo8/
foo8/Makefile
include/
lib/
and this top-level Makefile:
and this as each of the subdir-Makefiles:
Output:
Interestingly, it works again if any one of these actions is taken:
- the lib/ directory is removed
- the include/ directory is removed
- one of the entries of SUBDIRS is removed
- the line with vpath %.h include is removed
- the line with vpath %.h include is put after the vpath %.a ...
- in the line with vpath %.a ..., the entry lib is moved to the end of the list
|