bugmake - Bugs: bug #30340, dependency handling

 
 

bug #30340: dependency handling

Submitter:  None
Submitted:  Sun 04 Jul 2010 04:12:38 AM UTC
   
 
Severity:  3 - Normal Item Group:  Documentation
Status:  None Privacy:  Public
Assigned to:  None Open/Closed:  Open
Component Version:  3.81 Operating System:  Any
Fixed Release:  None Triage Status:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Mon 09 Aug 2010 12:11:57 PM UTC, comment #3: 

"When the .o file hasn't been created yet, knowing its .c source file is enough, and information on header file dependencies, while harmless, is of no practical use. "

Well, that's not always true. Some projects use generated headers and use makefile dependency tracking in order to build them before they are needed. Therefore sometimes it's essential to generate dependencies of foo.c just to know that we need to generate bar.h before compiling foo.o.

In case when you don't use generated headers you can as well bypass a rule for generating them and create dependencies as a compilation side-effect only. So your rule

%.o %.d: %.c
   $(COMPILE.c) -MD -o $*.o $<

would be enough to be just

%.o: %.c
   $(COMPILE.c) -MD -o $@ $<

You then include dependencies with

-include $(sources:.c=.d)

and only those dependencies which were already generated are evaluated.

Advantage of such approach is that dependencies are not generated if they are not needed (i.e. when calling make clean) but if they are present, they are used.

Krzysztof Malinowski <raspy>
Sun 04 Jul 2010 06:16:06 PM UTC, comment #2: 
Paul D. Smith <psmith>
Group administrator
Sun 04 Jul 2010 03:14:37 PM UTC, comment #1: 

I posted this bug, but forgot to login before posting.

Florian Rivoal <frivoal>
Sun 04 Jul 2010 04:12:38 AM UTC, original submission:  

In section 4.14, GNU make's info manual recommends the following construct to handle header files dependencies.

%.d: %.c
        @set -e; rm -f $@; \
        $(CC) -M $(CPPFLAGS) $< > $@.$$$$; \
        sed 's,\($*\)\.o[ :]*,\1.o $@ : ,g' < $@.$$$$ > $@; \
        rm -f $@.$$$$

sources = foo.c bar.c
include $(sources:.c=.d)

Indeed, this works, but I believe there are better solutions, both from an efficiency point of view, and from a pedagogic point of view. Let me give a couple of examples of what is less than ideal with this solution, and then show an alternative proposal.

1) .d files are generated and included even when they are not needed. When the .o file hasn't been created yet, knowing its .c source file is enough, and information on header file dependencies, while harmless, is of no practical use.

2) The rule for the %d: %c command is relatively complex, involving a sed expression that is likely to be cryptic for people in the process of learning the various GNU tools.

3) Generating the .d file and the .o file in completely independent steps is relatively inefficient, because it makes rather poor use of the file system's cache, and because it creates more processes than strictly necessary.

Instead I propose this:

%.o %.d: %.c
        $(COMPILE.c) -MD -o $*.o $<

sources = foo.c bar.c
objects = $(sources:.c=.o)
include $(patsubst %.o,%.d,$(wildcard $(objects)))


The advantages are:

Only the .d files matching existing objects are included. As these are the only .d files actually needed, this is more efficient, and addresses point 1.

There is no need to compute the dependencies of the .d file itself. As they are identical to the dependencies of the .o file, the .d file should be recreated every time the .o file is. The proposed rule takes care of that. This gets us rid of most of the complexity of the original rule for making the .d files, and in particular the sed command, addressing point 2.

Fewer process creations are needed. Instead of 4 per source file (cc for the .o file, cc for the .d file, sed, and rm), we have 1, addressing part of the efficiency concerns expressed in point 3.

The disc/file system is used more efficiently. The source file is only read once, causing much fewer disc accesses than the method proposed in GNU make's info manual. As compilation tends to be I/O bound, this can be significant. This also addresses the efficiency concerns expressed in point 3.

The only downside I see with this method is that when only the .d file is missing when the .o the other exists and is up to date, will we recreate both anyway, which is slightly inefficient. However, I don't consider this a big issue, because this will only happen in rare situations. As the two file are
generated together, in normal use it shouldn't happen often that only one of them is present. Besides, even if the .d is missing, the regeneration of the .o will not be that expensive, given that both files are generated with a single invocation of the compiler.

I don't know any compiler that supports -M and doesn't support -MD, but if there is one, the rule can be very simply adapted:

%.o %.d: %.c
        $(COMPILE.c) -o $*.o $<
        $(COMPILE.c) -M -o $*.d $<

This still creates fewer processes than the method proposed in GNU make's manual, and is still easier to understand. The two accesses to the source file are immediately after each other, and likely to play well with the file system's cache.

If you see problems with the method I suggest, I would be delighted to get your feedback on this topic. If you agree that the method I propose would be an improvement to the one currently described in the info manual, I will try to write a replacement for section 4.14, unless of course, you prefer to do that yourself.

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by raspy (Posted a comment)
  • -email is unavailable- added by psmith (Posted a comment)
  • -email is unavailable- added by frivoal (Posted a comment)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    No changes have been made to this item

    Back to the top

    Powered by Savane 3.13-4448.
    Corresponding source code