bugmake - Bugs: bug #58056, Forced prerequisite order is not...

 
 

bug #58056: Forced prerequisite order is not honored with pattern rules

Submitted by:  None
Submitted on:  Fri 27 Mar 2020 11:14:42 AM UTC  
 
Severity:  3 - Normal Item Group:  Bug
Status:  None Privacy:  Public
Assigned to:  None Open/Closed:  Open
Component Version:  4.3 Operating System:  None
Fixed Release:  None Triage Status:  None

Add a New Comment (Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

( Jump to the original submission)

Tue 31 Mar 2020 07:59:32 PM UTC, comment #14: 

In this bash session you can see when foo_A is missing make considers everything is up to date. Unlike when foo_B is missing. That's why make does not built foo_A during the initial travers. Because make considers foo_A an intermediate.

$ ls
makefile
$ cat makefile
all: foo.bar

%.bar: %_A %_B
        @echo $@

%_A:
        @echo $@

foo_B:
        @echo $@
$ make
foo_B
foo_A
foo.bar
$ touch foo_B foo.bar
$ ls foo_A
ls: cannot access foo_A: No such file or directory
$ make
gmake: Nothing to be done for `all'.
$ rm foo_B
$ make
foo_B
foo_A
foo.bar

Dmitry Goncharov <dgoncharov>
Mon 30 Mar 2020 08:20:48 PM UTC, comment #13: 

Note I haven't looked at the actual behavior you're reporting, of misordered recipes, and I haven't understood Dmitry's explanation.  Just looking at the behavior you see I do agree that it's wrong.  However there may be some underlying reason for it that I'm not remembering at the moment.

However, even if it were fixed to work as I think it should, that doesn't mean it works the way you seem to want.  In particular, if you invoked make with -j then make would definitely attempt to build all the prerequisites, both normal and order-only, at the same time.

The docs first describe the behavior of a normal prerequisite; in particular:

> first, it imposes an order in which recipes will be invoked: the
> recipes for all prerequisites of a target will be completed before
> the recipe for the target is run. Second, it imposes a dependency
> relationship: if any prerequisite is newer than the target, then
> the target is considered out-of-date and must be rebuilt.


This means that "foo: bar baz" ensures that both "bar" and "baz" will be complete before "foo" is invoked.  Note it says NOTHING about the relative order in which "bar" and "baz" will be invoked.  Secondarily, it says that if either "foo" or "bar" end up being more up to date than "foo", then "foo" will be rebuilt.

Next it discloses that an order-only prerequisite relaxes the second requirement:

> Occasionally, however, you have a situation where you want to impose
> a specific ordering on the rules to be invoked without forcing the
> target to be updated if one of those rules is executed.


This means that order-only prerequisites still adhere to the first behavior (both "bar" and "baz" are completed before "foo" is invoked), but they do not use the second behvior: just because "bar" or "baz" are newer than "foo", does NOT mean that "foo" will be rebuilt.

None of this entire section discusses any dependency between two prerequisites of the same rule.  All issues of ordering here are discussing ordering between the target and its prerequisites.

There is, in fact, no way to order the prerequisites of a target from within the list of prerequisites itself.  If you want to impose ordering you must do it in a separate rule, where the target will be built after its prerequisites.

Paul D. Smith <psmith>
Project Administrator
Mon 30 Mar 2020 07:35:01 PM UTC, comment #12: 

If this behavior is allowed, I think the documentation should clarify what the order-only prerequisites actually means. I still haven't figured out their purpose or how they operate based on the documentation or what has been said here. To me, the documentation basically implies that everything after the '|' is serialized.

Sorry if this is getting of topic, but the issue exists in the first place because of this confusion. I usually never assume any serialization of the prerequisites, but I needed to enforce it in a particular scenario and though the '|' was for this.

Anonymous
Mon 30 Mar 2020 05:00:29 PM UTC, comment #11: 

Well, building intermediate prerequisites during this initial travers along with regular prerequisites is wasteful, if all regular prerequisites are up to date.
If order matters then one prerequisite should explicitly depend on another.

Dmitry Goncharov <dgoncharov>
Mon 30 Mar 2020 04:55:09 PM UTC, comment #10: 

It's true that POSIX can't require anything of pattern rules since they are not part of POSIX.  But that doesn't mean we shouldn't try to make them behave the same way, as a matter of QoI.

Paul D. Smith <psmith>
Project Administrator
Mon 30 Mar 2020 04:47:03 PM UTC, comment #9: 

i believe make complies with this requirement.
This out of order behavior is caused by mixing intermediate prerequisites and prerequisites in the same rule.
Posix does not specify intermediate prerequisites such as %_A.
Once your makefile has such prerequisites you are outside the posix realm.

Dmitry Goncharov <dgoncharov>
Mon 30 Mar 2020 04:36:11 PM UTC, comment #8: 

There may be some confusion.  Order-only prerequisites have no implied ordering amongst themselves, which is what Dmitry meant.

That is, if you have:

foo: bar biz | baz boz

then all the prerequisites bar, biz, baz, boz will be completed before foo is run.  However, this does not say that baz will be completed before boz is run (or any of the other prerequisites).

However I'm not convinced this has anything to do with the issue you've reported.  I"m with Sam this is a red herring.

There is indeed an order of processing which is guaranteed by make when run serially, mandated by the POSIX spec as well, which says:

> The make utility shall treat all prerequisites as targets
> themselves and recursively ensure that they are up-to-date,
> processing them in the order in which they appear in the rule.

Paul D. Smith <psmith>
Project Administrator
Mon 30 Mar 2020 04:34:17 PM UTC, comment #7: 

comment #6:

> Wait, so order-only prerequisites does NOT mean serialized make for these prerequisites? "a situation where you want to impose a specific ordering on the rules to be invoked" ← That's not what this means?


Right. That is not what it means. The ordering it's talking about is that all the prerequisite recipes will run before the target recipe, if indeed the target recipe runs at all. Nothing about serializing the prerequisite recipes. Nothing about the relative order of the prequisite recipes.

But again, we are somewhat off-topic.

Sam Kendall <sckendall>
Mon 30 Mar 2020 04:17:24 PM UTC, comment #6: 

comment #4:

> So I claim that parallel make and order-only prerequisites are distractions here. We should start with the documentation: what does it guarantee about the build order of the prereqs in the example makefile variants, in a serial make?


Wait, so order-only prerequisites does NOT mean serialized make for these prerequisites? "a situation where you want to impose a specific ordering on the rules to be invoked" ← That's not what this means?

Anonymous
Mon 30 Mar 2020 04:12:07 PM UTC, comment #5: 

comment #3:

> make builds order only prerequisites in parallel.


How can the ordering on the rules be respected then?

Anonymous
Mon 30 Mar 2020 04:11:39 PM UTC, comment #4: 

I think we can assume the original report was about serial execution of make. Also, if I remove the vertical bar from the example makefile (or from the variation with "foo_A" instead of "%_A"), I get the identical behavior. I tried versions 3.82 and 4.3.

So I claim that parallel make and order-only prerequisites are distractions here. We should start with the documentation: what does it guarantee about the build order of the prereqs in the example makefile variants, in a serial make?

I couldn't find any such guarantees. I've seen countless (serial) makefiles that count on prereqs building serially, e.g., "do_something: step1 step2 step3". Apparently we can't count on that order if one of the prereqs matches a pattern rule; but I didn't find anything about that, either.

The GNU make doc is generally very clear and complete. I suspect I'm missing something.

Sam Kendall <sckendall>
Mon 30 Mar 2020 03:22:57 PM UTC, comment #3: 

make builds order only prerequisites in parallel.

Dmitry Goncharov <dgoncharov>
Mon 30 Mar 2020 02:11:29 PM UTC, comment #2: 

comment #1:

> This behavior is intended.
> [...]
> 2. Another reason is to allow parallel execution. If all prerequisites had to be built in the specified order how would you built them in parallel?


Did you notice the '|'? foo_A and foo_B are order-only-prerequisites, which should disable any parallelism.

https://www.gnu.org/software/make/manual/html_node/Prerequisite-Types.html

Anonymous
Sun 29 Mar 2020 02:27:08 AM UTC, comment #1: 

This behavior is intended.

1. To figure out if a target has to be rebuilt make traverses the list of prerequisites and finds out if any of the prerequisites, but not intermediate prerequisites, need to be rebuilt.
During this traverse make rebuilds all prerequisites that are out of date. If, during this traverse, make finds out that some prerequisite was out of date, then make proceeds to rebuild intermediate prerequisites.

Intermediate prerequisites are not rebuild during this traverse along with prerequisites because, if all prerequisites are up to date then the target is up to date and there is no need to built intermediate prerequisites.

In your example
foo_A is an intermediary prerequisite.
foo_B is a prerequisite.

2. Another reason is to allow parallel execution. If all prerequisites had to be built in the specified order how would you built them in parallel?

Dmitry Goncharov <dgoncharov>
Fri 27 Mar 2020 11:14:42 AM UTC, original submission:  

Given the following Makefile:

all: foo.bar

%.bar: | %_A %_B
        @

%_A:
        @echo $@
foo_B:
        @echo $@

We get:

% make
foo_B
foo_A

Instead of:

% make
foo_A
foo_B

The forced order "%_A %_B" in the prerequisites of "%.bar" is not respected.

Note that replacing "foo_B:" with "%_B" does not trigger the bug.

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by psmith (Posted a comment)
  • -email is unavailable- added by sckendall (Posted a comment)
  • -email is unavailable- added by dgoncharov (Posted a comment)
  •  

    Do you think this task is very important?
    If so, you can add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

     

     

    No changes have been made to this item

    Back to the top


    Powered by Savane 3.5