bugmake - Bugs: bug #19108, Pattern rules with multiple target...

 
 

bug #19108: Pattern rules with multiple target patterns do not honor the dependencies of all targets correctly

Submitted by:  Christoph Schulz <kristovschulz>
Submitted on:  Tue 20 Feb 2007 01:45:23 PM UTC  
 
Severity:  3 - Normal Item Group:  Bug
Status:  Fixed Privacy:  Public
Assigned to:  Paul D. Smith <psmith> Open/Closed:  Closed
Component Version:  3.81 Operating System:  Any
Fixed Release:  3.82 Triage Status:  None

Add a New Comment (Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

( Jump to the original submission)

Tue 01 Jun 2010 12:49:32 PM UTC, comment #7: 

This problem (in comment #6 ) is not related to this bug in any way.  Please ask your question on one of the mailing lists such as -email is unavailable-

Paul D. Smith <psmith>
Project AdministratorIn charge of this item.
Tue 01 Jun 2010 11:38:45 AM UTC, comment #6: 

I am lost. In my case 2 targets are generated after a command is completed.

targ1 targ2: some dependencies
      ./my_long_command

And I try to avoid two invocations of the command. When I use "make -j 2" the double invocation happens. Since my command lasts 2 hours it becomes an issue :(

How to avoid double invocation?
Any hints/comments?

Valery <vak>
Wed 10 Jun 2009 02:09:54 AM UTC, comment #5: 

When we are remaking the deps for a given file, we also have to try to remake all the deps of all the also_make targets of that file, right then to be sure they're ordered properly.

Paul D. Smith <psmith>
Project AdministratorIn charge of this item.
Fri 22 Jun 2007 03:36:12 PM UTC, comment #4: 

My Comment (#3 below) may be bogus; I thought I was defining a pattern rule (which supports one-action, multiple targets), but it turned out I hadn't (so the rule didn't support multiple targets).

Stephen Warren <srwarren>
Thu 21 Jun 2007 06:15:16 PM UTC, comment #3: 

I think I'm seeing some symptoms that are essentially the same bug. I'm using make 3.80.

I have a rule that generates N targets from some dependencies using a set of commands, setup just like the original poster.

If I run "make", everything is fine; the commands are run once, and make knows that all the N targets have been updated just fine.

However, if I run "make -j 2", make spawns off two copies of the commands, presumably because when it's attempting to build each of the N targets, it doesn't check the "make_also" to see if some other target of this rule is already being built. Once a target has finished building, it marks all the "make_also" as being built, and so doesn't spawn the commands again for the other N-2 targets.

Probably, the original reporter's problem is also a lack of checking make_also, but in this case, a lack of checking the dependencies rather than the build-in-progress value of the make_also targets.

Anonymous
Thu 24 May 2007 10:32:38 PM UTC, comment #2: 

Hello Krzysztof!

> [...] so it calls your rules to build x.tgt1 ONLY


Yes, but this behaviour is not correct. Currently, pattern rules with multiple target patterns are the only way to specify that a tool is generating multiple output files at once. This is clearly stated in the documentation (see 10.5.1). So GNU make already knows that executing the pattern rule will generate all the targets of that rule, not only the target GNU make wants to update. Consequently, GNU make should consider that and ensure that the prerequisites of all the other targets that are also created (unavoidably) by the pattern rule are built before the rule is executed. Currently, this is not done, and although this behaviour is documented, I don't think it is correct.

> However this single rule should NOT generate both files at
> once.


That's what pattern rules with multiple target patterns are for, see 10.5.1.

> My belief is that no rule should make more targets then make
> wants.


Yes, but if you use a pattern rule with multiple target patterns, this situation is unavoidable, as the command being executed cannot be forced to create only a subset of its output files in general. (My example is, of course, only a simplification of a more complex real-world situation.)

Regards,
  Christoph Schulz

Christoph Schulz <kristovschulz>
Thu 24 May 2007 10:09:38 AM UTC, comment #1: 

Hello Christoph,

I believe that the problem lies in your Makefile rule:

%.tgt1 %.tgt2: %.src
cp $< $(patsubst %.src,%.tgt1,$<)
cp $< $(patsubst %.src,%.tgt2,$<)

This rule states that %.tgt1 depends on %.src and %.tgt2 depends on %.src. However this single rule should NOT generate both files at once. I believe that if you rewrite your rule as follows:

%.tgt1 %.tgt2: %.src
cp $< $@

then it would work as desired.

The issue is that if multiple targets pattern rule is defined, make runs commands for one fitting target at the time. Now, when you build "final", make sees that it depends on "x". "x" depends on x.tgt1, which can be built due to no further dependencies, so it calls your rules to build x.tgt1 ONLY. But your rule builds also x.tgt2, so when make comes back to check other dependencies of x, x.tgt2 is already up-to-date and does not need to be followed.

My belief is that no rule should make more targets then make wants. Every rule should only generate $@. Otherwise one is asking for trouble.

Best regards,
Krzysztof Malinowski

Anonymous
Tue 20 Feb 2007 01:45:23 PM UTC, original submission:  

[Tested on Windows with native (non-MSYS) MinGW build of GNU make 3.81, found in http://prdownloads.sourceforge.net/mingw/mingw32-make-3.81-1.tar.gz, and on Gentoo Linux with portage-compiled make 3.81.]

Hello,

I think I've found a bug in GNU make. When a pattern rule with multiple targets exists, and make determines that it needs that rule to built one matching target, it uses this rule even if the other targets implicitly built by this rule don't have their dependencies built yet.

As this is quite complicated to explain, I created a little example:

final: x
@echo "making final"
touch $@
x: x.tgt1 x.tgt2
@echo "making x"
touch $@
x.tgt2: dep
dep:
@echo "making dep"
sleep 5
touch $@
%.tgt1 %.tgt2: %.src
cp $< $(patsubst %.src,%.tgt1,$<)
cp $< $(patsubst %.src,%.tgt2,$<)
clean:
-rm -f final x dep x.tgt1 x.tgt2

(Note: To run this makefile successfully you have to create a file named "x.src" at first, to cause the pattern rule to be applied.)

This Makefile has some unwanted properties. At first, building "final" does not result in building "dep", although the dependency chain "final-->x-->x.tgt2-->dep" exists. Make traverses the dependency tree from left to right. It finds that x depends on x.tgt1 and that x.tgt1 can only be built by the pattern rule. However, at build time, this rule also builds x.tgt2. So x.tgt2's non-existent dependencies are ignored as it already exists.

If you reverse the order of x's prerequisites, you encounter an even stranger effect. Running make (after a proper "make clean", of course) seems to work. However, if you use "make -j" you will see that the "dep" target will be finished long after the "final" target. This is due to the fact that the dependency x.tgt2-->dep is not properly tracked across the pattern rule.

Additionally, in both cases (Makefile with original order of x's
prerequisites and Makefile with reversed order and using "make -j"), one has to run make a second time until everything is built.

I don't know whether this behaviour is by design. But I think that pattern rules with multiple (pattern) targets must only be run if make determines that all dependencies for all the targets the pattern rule would create are already in place. Currently, make seems to perform the dependency check only for that target that triggers the rule. This is not enough IMHO. In the example above, make should be running the pattern rule only after the dependencies of both x.tgt1 and x.tgt2 have been built successfully. This would also help to get rid of the "reordering changes semantics" effect described above.

The background: The real case was a system where the compiler (MSVC++) also creates a precompiled header file for a special dummy source file. To account for the fact that compiling all "normal" source files needs this precompiled headers I built a pattern rule like the following one (the pattern rule was necessary in order to tell "make" that the command build both files simultaneously):

OBJS := library-pchgen.obj <further library objects>
$(OBJS): library-pchgen.obj

%.pch %-pchgen.obj: %-pchgen.src
  <compiler call to compile the pchgen and pch files>

$(TARGET): $(OBJS) library.pch
  <linker call>

Because there were dependencies on other libraries, all the object files had additional prerequisites:

$(OBJS): other_lib/other_lib.dll another_lib/another_lib.dll

However, this did not work when using "make -j", as library-pchgen.obj was built before the other libraries were built. Stating explicitly that the pch file (library.pch) also depends on the other libraries via:

$(OBJS) library.pch: other_lib/other_lib.dll another_lib/another_lib.dll

solved the problem, but - as I already said - I don't think this should be necessary.)

Regards,
  Christoph Schulz

Christoph Schulz <kristovschulz>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #12019:  Makefile added by kristovschulz (290B - application/octet-stream - Testcase)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by vak (Posted a comment)
  • -email is unavailable- added by psmith (Posted a comment)
  • -email is unavailable- added by srwarren (Posted a comment)
  • -email is unavailable- added by kristovschulz (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

     

     

    Follow 7 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2010-07-28 psmith Fixed Release4.0 => 3.82
    2009-06-10 psmith StatusNone => Fixed
        Assigned toNone => psmith
        Open/ClosedOpen => Closed
        Fixed ReleaseNone => 4.0
    2007-06-21 srwarren Carbon-Copy- => Added srwarren
    2007-02-20 kristovschulz Attached File- => Added Makefile, #12019

    Back to the top


    Powered by Savane 3.5