bugmake - Bugs: bug #51462, Double-colon dependencies are...

 
 

bug #51462: Double-colon dependencies are built serially with parallel make

Submitter:  None
Submitted:  Thu 13 Jul 2017 09:29:18 PM UTC
   
 
Severity:  3 - Normal Item Group:  Bug
Status:  None Privacy:  Public
Assigned to:  None Open/Closed:  Open
Component Version:  4.3 Operating System:  None
Fixed Release:  None Triage Status:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Thu 13 Jul 2017 10:13:54 PM UTC, comment #2: 

Thanks.

I suspect that the fix referred to in 47995 is:

http://git.savannah.gnu.org/cgit/make.git/commit/?id=4762480ae9cb8df4878286411f178d32db14eff0

(it even starts with "[SV 47995]").

However, no, unfortunately it doesn't fix the problem I described.

Robert Morell <rmorell>
Thu 13 Jul 2017 10:07:26 PM UTC, comment #1: 

A google for that hash finds:

https://savannah.gnu.org/bugs/?48057

... in which a reversion of that commit is suggested.  It's marked as a duplicate of:

https://savannah.gnu.org/bugs/?47995

... which is marked as fixed in 4.2.1, which is the version in which you had the problem.  So that's odd, then.  Don't you wish that the commits posted to the bug tracker, so that you could trivially see what was done to fix the bug?

Martin Dorey <mdorey>
Thu 13 Jul 2017 09:29:18 PM UTC, original submission:  

I noticed that some builds are significantly slower after switching from make 4.0 to 4.2.1.  I tracked it down to what seems to be a parallel make problem with double-colon prerequisites.  In particular, when there are multiple double-colon prereqs for a target, each of those prereqs seems to be considered serially, although their dependencies are built in parallel.  It may be easier to explain with an example.

Test makefile:


default: all

# define a rule to build 'file$(1)', as an intermediate for 'target$(2)'.
#
# The recipe just prints the time, target name, and information about what
# target it's for, sleeps for $(1) seconds, then touches the file.
#
# It also adds a dependency for 'target$(2)'.
define FILE_RULE

file$(1):
        @echo At `date +%k:%M:%S` building $$@, for $(2)
        @sleep $(1)
        @touch $$@

target$(2): file$(1)

endef

# define a rule to build 'target$(1)'.
#
# The recipe just prints the time and touches the file.
#
# It also adds a double-colon dependency for the 'all' phony target, so that
# 'target$(1)' will get built when 'all' is the goal.
define TARGET_RULE

target$(1):
        @echo At `date +%k:%M:%S` building $$@
        @touch $$@

all:: target$(1)

endef

# Declare a few arbitrary dependency trees using the convenience macros above.
# The first two have three dependent files each, the last one has six.
$(eval $(call FILE_RULE,1,1))
$(eval $(call FILE_RULE,2,1))
$(eval $(call FILE_RULE,3,1))
$(eval $(call TARGET_RULE,1))

$(eval $(call FILE_RULE,4,2))
$(eval $(call FILE_RULE,5,2))
$(eval $(call FILE_RULE,6,2))
$(eval $(call TARGET_RULE,2))

$(eval $(call FILE_RULE,7,3))
$(eval $(call FILE_RULE,8,3))
$(eval $(call FILE_RULE,9,3))
$(eval $(call FILE_RULE,10,3))
$(eval $(call FILE_RULE,11,3))
$(eval $(call FILE_RULE,12,3))
$(eval $(call TARGET_RULE,3))

clean:
        rm -rf file* target*

.PHONY: all clean


With GNU make 4.0 the output of 'time make -j' after 'make clean' is:

At 14:08:02 building file4, for 2
At 14:08:02 building file6, for 2
At 14:08:02 building file3, for 1
At 14:08:02 building file2, for 1
At 14:08:02 building file1, for 1
At 14:08:02 building file5, for 2
At 14:08:02 building file9, for 3
At 14:08:02 building file11, for 3
At 14:08:02 building file7, for 3
At 14:08:02 building file12, for 3
At 14:08:02 building file8, for 3
At 14:08:02 building file10, for 3
At 14:08:05 building target1
At 14:08:08 building target2
At 14:08:14 building target3

real        0m12.026s
user        0m0.008s
sys        0m0.004s


Note that all 'fileN' recipes begin at the same time, and the dependent 'targetN' recipes begin as soon as their respective prereqs are complete.

The same with GNU make 4.2.1:

At 14:09:25 building file1, for 1
At 14:09:25 building file2, for 1
At 14:09:25 building file3, for 1
At 14:09:28 building target1
At 14:09:28 building file4, for 2
At 14:09:28 building file5, for 2
At 14:09:28 building file6, for 2
At 14:09:34 building target2
At 14:09:34 building file7, for 3
At 14:09:34 building file8, for 3
At 14:09:34 building file9, for 3
At 14:09:34 building file10, for 3
At 14:09:34 building file11, for 3
At 14:09:34 building file12, for 3
At 14:09:46 building target3

real        0m21.052s
user        0m0.004s
sys        0m0.004s


Note how the prereqs for target1 are built, then once that's complete the prereqs for target2, and then finally target3.  Each set doesn't start until the previous has completed.


I bisected the GNU make repository and found that this is caused by the commit:
[SV 44742] Fix double-colon rules plus parallel builds.
(9bb994e8319c2b153cd3d6d61e2c2882895e7c3a in the git repo).

The 4.2.1 behavior also occurs at the latest commit from source control (e2ebea35f11059e in the git repo).

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by rmorell (Posted a comment)
  • -email is unavailable- added by mdorey (Posted a comment)
  • -email is unavailable- added by rmorell
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follows 1 latest change.

    Date Changed by Updated Field Previous Value => Replaced by
    2017-07-13 rmorell Carbon-Copy- Added rmorell

    Back to the top

    Powered by Savane 3.13-f8d8.
    Corresponding source code