bugmake - Bugs: bug #54854, multi-target rules invoked too...


bug #54854: multi-target rules invoked too often with -j2

Submitted by:  Bernhard M. <bmwiedemann>
Submitted on:  Wed 17 Oct 2018 11:08:15 AM UTC  
Severity:  3 - Normal Item Group:  None
Status:  Not A Bug Privacy:  Public
Assigned to:  None Open/Closed:  Closed
Component Version:  None Operating System:  None
Fixed Release:  None Triage Status:  None

Add a New Comment (Rich Markup)

You are not logged in

Please log in, so followups can be emailed to you.


Wed 17 Oct 2018 01:19:36 PM UTC, comment #1: 

This is not a bug, this is how every version of make has always worked, since make was invented.

Your reproducer is not really minimal :).  Here's a minimal reproducer:

all: foo.1 bar.1 biz.1
foo.1 bar.1 biz.1: ; touch foo.1 bar.1 biz.1

When you write a rule with three targets that's exactly the same thing as writing three separate rules; see https://www.gnu.org/software/make/manual/html_node/Multiple-Targets.html :

all: foo.1 bar.1 biz.1
foo.1: ; touch foo.1 bar.1 biz.1
bar.1: ; touch foo.1 bar.1 biz.1
baz.1: ; touch foo.1 bar.1 biz.1

If you run in parallel make will run all of these rules at the same time.  It has no way of knowing that one of these recipes will build all three targets.  If you run serially than make still would consider running all recipes, except that it sees that the other targets have been updated after the first one is complete by looking at the modification times.

You can solve this problem by using pattern rules:

all: foo.1 bar.1 biz.1
foo.% bar.% baz.%: ; touch foo.1 bar.1 biz.1

because the way make treats pattern rules with multiple targets is different: see https://www.gnu.org/software/make/manual/html_node/Pattern-Intro.html

Or you can rewrite your rules to build one target per invocation rather than building lots of targets at once: this is how make was designed to work.

If you'd like to discuss further please open a thread at the -email is unavailable- or -email is unavailable- mailing lists.

Paul D. Smith <psmith>
Project Administrator
Wed 17 Oct 2018 11:08:15 AM UTC, original submission:  

Seen in the wild at https://github.com/github/hub/pull/1892
I made a minimal reproducer (attached).

To reproduce:
tar xf makebug.tar.gz
cd makebug/
make clean ; time make -j2
make clean ; time make -j1

notice how the -j2 takes longer than -j1
because in make -j1, the multi-target rule to create all man-pages is just invoked once, but with -j2 it is invoked three times - once per input file, not only wasting compute resources, but also creating a chance for race conditions.

Maybe there is a better (parallelism-safe) way to specify such cases where one command turns many input files into many output files than the bison-example at https://www.gnu.org/software/make/manual/make.html#Pattern-Examples ?

Bernhard M. <bmwiedemann>


(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:

Attached Files
file #45219:  makebug.tar.gz added by bmwiedemann (16KiB - application/gzip)


Depends on the following items: None found

Items that depend on this one: None found


Carbon-Copy List
  • -email is unavailable- added by psmith (Posted a comment)
  • -email is unavailable- added by bmwiedemann (Submitted the item)

    Do you think this task is very important?
    If so, you can add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.




    Follow 3 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2018-10-17 psmith StatusNone => Not A Bug
        Open/ClosedOpen => Closed
    2018-10-17 bmwiedemann Attached File- => Added makebug.tar.gz, #45219

    Back to the top

    Powered by Savane 3.6