bugmake - Bugs: bug #51462, Double-colon dependencies are...

 
 

bug #51462: Double-colon dependencies are built serially with parallel make

Submitter:  None
Submitted:  Thu 13 Jul 2017 09:29:18 PM UTC
   
 
Severity:  3 - Normal Priority:  5 - Normal
Item Group:  Bug Status:  None
Privacy:  Public Assigned to:  None
Open/Closed:  Open Component Version:  4.3
Operating System:  None Fixed Release:  None
Triage Status:  None
* Mandatory Fields

Post a Comment

Add a New Comment Rich Markup
   

Discussion

Thu 13 Jul 2017 10:13:54 PM UTC, comment #2: 

Thanks.

I suspect that the fix referred to in 47995 is:

http://git.savann ... 11f178d32db14eff0

(it even starts with "[SV 47995]").

However, no, unfortunately it doesn't fix the problem I described.

Robert Morell <rmorell>
Thu 13 Jul 2017 10:07:26 PM UTC, comment #1: 

A google for that hash finds:

https://savannah. ... u.org/bugs/?48057

... in which a reversion of that commit is suggested.  It's marked as a duplicate of:

https://savannah. ... u.org/bugs/?47995

... which is marked as fixed in 4.2.1, which is the version in which you had the problem.  So that's odd, then.  Don't you wish that the commits posted to the bug tracker, so that you could trivially see what was done to fix the bug?

Martin Dorey <mdorey>
Thu 13 Jul 2017 09:29:18 PM UTC, original submission:  

I noticed that some builds are significantly slower after switching from make 4.0 to 4.2.1.  I tracked it down to what seems to be a parallel make problem with double-colon prerequisites.  In particular, when there are multiple double-colon prereqs for a target, each of those prereqs seems to be considered serially, although their dependencies are built in parallel.  It may be easier to explain with an example.

Test makefile:


default: all

# define a rule to build 'file$(1)', as an intermediate for 'target$(2)'.
#
# The recipe just prints the time, target name, and information about what
# target it's for, sleeps for $(1) seconds, then touches the file.
#
# It also adds a dependency for 'target$(2)'.
define FILE_RULE

file$(1):
        @echo At `date +%k:%M:%S` building $$@, for $(2)
        @sleep $(1)
        @touch $$@

target$(2): file$(1)

endef

# define a rule to build 'target$(1)'.
#
# The recipe just prints the time and touches the file.
#
# It also adds a double-colon dependency for the 'all' phony target, so that
# 'target$(1)' will get built when 'all' is the goal.
define TARGET_RULE

target$(1):
        @echo At `date +%k:%M:%S` building $$@
        @touch $$@

all:: target$(1)

endef

# Declare a few arbitrary dependency trees using the convenience macros above.
# The first two have three dependent files each, the last one has six.
$(eval $(call FILE_RULE,1,1))
$(eval $(call FILE_RULE,2,1))
$(eval $(call FILE_RULE,3,1))
$(eval $(call TARGET_RULE,1))

$(eval $(call FILE_RULE,4,2))
$(eval $(call FILE_RULE,5,2))
$(eval $(call FILE_RULE,6,2))
$(eval $(call TARGET_RULE,2))

$(eval $(call FILE_RULE,7,3))
$(eval $(call FILE_RULE,8,3))
$(eval $(call FILE_RULE,9,3))
$(eval $(call FILE_RULE,10,3))
$(eval $(call FILE_RULE,11,3))
$(eval $(call FILE_RULE,12,3))
$(eval $(call TARGET_RULE,3))

clean:
        rm -rf file* target*

.PHONY: all clean


With GNU make 4.0 the output of 'time make -j' after 'make clean' is:

At 14:08:02 building file4, for 2
At 14:08:02 building file6, for 2
At 14:08:02 building file3, for 1
At 14:08:02 building file2, for 1
At 14:08:02 building file1, for 1
At 14:08:02 building file5, for 2
At 14:08:02 building file9, for 3
At 14:08:02 building file11, for 3
At 14:08:02 building file7, for 3
At 14:08:02 building file12, for 3
At 14:08:02 building file8, for 3
At 14:08:02 building file10, for 3
At 14:08:05 building target1
At 14:08:08 building target2
At 14:08:14 building target3

real        0m12.026s
user        0m0.008s
sys        0m0.004s


Note that all 'fileN' recipes begin at the same time, and the dependent 'targetN' recipes begin as soon as their respective prereqs are complete.

The same with GNU make 4.2.1:

At 14:09:25 building file1, for 1
At 14:09:25 building file2, for 1
At 14:09:25 building file3, for 1
At 14:09:28 building target1
At 14:09:28 building file4, for 2
At 14:09:28 building file5, for 2
At 14:09:28 building file6, for 2
At 14:09:34 building target2
At 14:09:34 building file7, for 3
At 14:09:34 building file8, for 3
At 14:09:34 building file9, for 3
At 14:09:34 building file10, for 3
At 14:09:34 building file11, for 3
At 14:09:34 building file12, for 3
At 14:09:46 building target3

real        0m21.052s
user        0m0.004s
sys        0m0.004s


Note how the prereqs for target1 are built, then once that's complete the prereqs for target2, and then finally target3.  Each set doesn't start until the previous has completed.


I bisected the GNU make repository and found that this is caused by the commit:
[SV 44742] Fix double-colon rules plus parallel builds.
(9bb994e8319c2b153cd3d6d61e2c2882895e7c3a in the git repo).

The 4.2.1 behavior also occurs at the latest commit from source control (e2ebea35f11059e in the git repo).

Anonymous

 

Attached Files

This item currently has no attached files.

(Note: upload size limit is set to 4.0MiB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

 

Dependencies

This item does not depend on any other items.

No items depend on this one.

 

Mail Notification Carbon-Copy List

Carbon-Copy List
  • -email is unavailable- added by rmorell (Posted a comment)
  • -email is unavailable- added by mdorey (Posted a comment)
  • -email is unavailable- added by rmorell
  •  

    Votes

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

    History

    Follows 1 latest change.

    Date Changed by Updated Field Previous Value => Replaced by
    2017-07-13 rmorell Carbon-Copy- Added rmorell

    Back to the top

    Powered by Savane 3.16-2753.
    Corresponding source code