bugmake - Bugs: bug #63125, Automatic variables in Secondary...

 
 

bug #63125: Automatic variables in Secondary Expansion don't work as documented

Submitter:  Riccardo P. Bestetti <vmsh0>
Submitted:  Wed 28 Sep 2022 08:24:02 PM UTC
   
 
Severity:  3 - Normal Item Group:  Enhancement
Status:  None Privacy:  Public
Assigned to:  None Open/Closed:  Open
Component Version:  4.3 Operating System:  POSIX-Based
Fixed Release:  None Triage Status:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Sat 15 Oct 2022 10:44:56 PM UTC, comment #6: 

I'm going to change this into an enhancement, to figure out how to make this comprehensible and "do the right thing" then whatever that is, document it clearly.

Paul D. Smith <psmith>
Group administrator
Thu 29 Sep 2022 06:09:06 PM UTC, comment #5: 

Ok, this is very clear to me now. Thank you for your examples.

Riccardo P. Bestetti <vmsh0>
Thu 29 Sep 2022 05:50:14 PM UTC, comment #4: 

Interestingly, if you switch the $$< to the other prerequisite then the newer releases do the right thing (completely), and GNU make 3.81 expands to empty:

$ cat Makefile
.SECONDEXPANSION:

foo: foo.1 $$<
foo: foo.2 ; : '<=$<' '+=$+'

$ make-4.3
: foo.2
: foo.1
: '<=foo.2' '+=foo.2 foo.1 foo.2'

$ make-3.81
: foo.2
: foo.1
: '<=foo.2' '+=foo.2 foo.1'

Note how here, GNU make 4.3 does the right thing by expanding $$< to the correct value of $<, foo.2, and adding that to the prerequisites list.

But, GNU make 3.81 doesn't even add the wrong thing here: it adds nothing.

Paul D. Smith <psmith>
Group administrator
Thu 29 Sep 2022 05:45:24 PM UTC, comment #3: 


> Well, it did work that way on Make 3.81.

It did something in GNU make 3.81, but it wasn't the right thing.  I tried to show this in my previous answer but I was not very clear about it, sorry about that.

Consider this output, without .SECONDEXPANSION, which you will get for ALL versions of GNU make:

$ cat Makefile
foo: foo.1
foo: foo.2 ; : '<=$<' '+=$+'

foo.%: ; : $@

$ make
: foo.2
: foo.1
: '<=foo.2' '+=foo.2 foo.2'

Note carefully here that although foo: foo.1 appears FIRST in the makefile, the value of $< is actually foo.2, not foo.1, and foo.2 is built first followed by foo.1.  That's because foo.2 is the first prerequisite in the rule where the recipe appears, and that's how the first prerequisite is always chosen (see the docs for this).

Now when we add in .SECONDEXPANSION, we can see that actually, GNU make 3.81 gets this wrong:

$ cat Makefile
.SECONDEXPANSION:

foo: foo.1
foo: foo.2 $$< ; : '<=$<' '+=$+'

foo.%: ; : $@

$ make
: foo.2
: foo.1
: '<=foo.2' '+=foo.2 foo.1 foo.1'

Note here that although the value if $< is foo.2 as it should be, actually when we expanded $$< in the secondary expansion it resolved to foo.1, because at that time it seemed like foo.1 was the first prerequisite: we had not yet parsed the current prerequisite list so we didn't replace it with foo.2.

So although GNU make 3.81 did replace $$< with some value, not an empty value, it was not actually the right value.

Paul D. Smith <psmith>
Group administrator
Thu 29 Sep 2022 12:10:21 AM UTC, comment #2: 

comment #1:

> Also, recall that $^ shows a set of prerequisites that does NOT include duplicates.  If you want to see the full set of prerequisites including duplicates, you need to use $+.


So sorry - my testing, and the output that is shown in my original submission, indeed uses $+ and not $^ for both make 3.81 and make 4.3. I just copied over the wrong Makefile when opening the bug report. I re-run the correct Makefile ($+) and I confirm the outputs are correctly reported.

>
> However even if you use $+ you'll get a different answer because indeed $$< is not set here.  There seems to be some dissension between the docs and the code, which we should investigate.
>
> In general it can be tricky to use automatic variables that contain prerequisites, in the list of prerequisites, because there's a catch-22 there.  Resolving it would require that each prerequisite be parsed and all automatic variables be updated before any subsequent secondary expansion is done and I'm not sure that's how it works.


Well, it did work that way on Make 3.81. (At least, apparently, with $$< - that, being limited to representing the first prerequisite, doesn't create any infinite recursion or paradox if NOT used as the first prerequisite itself.)

>
> FYI I checked and the behavior you see in 4.3 was the same as in 3.82.

I can also confirm that.

Riccardo P. Bestetti <vmsh0>
Wed 28 Sep 2022 08:48:35 PM UTC, comment #1: 

A result of foo.1 is definitely not right.

The value of $< is always set to the first prerequisite in the rule definition which provides the recipe, if there is one.  Only if there isn't a prerequisite listed in the rule itself will you get the first found prerequisite.  See:


$ cat Makefile
foo: foo.1
foo: foo.2 ; : $@ $<

foo.%: ; : $@

bar: foo.3
bar: foo.4
bar: ; : $@ $<

$ make foo
: foo.2
: foo.1
: foo foo.2

$ make bar
: foo.3
: foo.4
: bar foo.3


We can see that the behavior in GNU make 3.81 was definitely wrong.

Also, recall that $^ shows a set of prerequisites that does NOT include duplicates.  If you want to see the full set of prerequisites including duplicates, you need to use $+.

However even if you use $+ you'll get a different answer because indeed $$< is not set here.  There seems to be some dissension between the docs and the code, which we should investigate.

In general it can be tricky to use automatic variables that contain prerequisites, in the list of prerequisites, because there's a catch-22 there.  Resolving it would require that each prerequisite be parsed and all automatic variables be updated before any subsequent secondary expansion is done and I'm not sure that's how it works.

FYI I checked and the behavior you see in 4.3 was the same as in 3.82.

Paul D. Smith <psmith>
Group administrator
Wed 28 Sep 2022 08:24:02 PM UTC, original submission:  

The GNU Make manual on Secondary Expansion[1] reports in section "Secondary Expansion of Explicit Rules":

"The $$< variable evaluates to the first prerequisite in the first rule for this target."

As such, with the Makefile reported at the end of this submission (Makefile), I would expect the output "foo.1 foo.2 foo.1" or "foo.2 foo.1 foo.1". Instead, I get the output "foo.1 foo.2".

Note that this doesn't only work unexpectedly for the $$< second expansion automatic variable, but also for the other automatic variables, which also expand to the empty string.

See Exhibit 1 and Exhibit 2, both at the end of this submission, to observe that the unexpected behaviour doesn't present e.g. in Make 3.81.

[1]: https://www.gnu.org/software/make/manual/html_node/Secondary-Expansion.html



Makefile

.SECONDEXPANSION:

foo: foo.1
foo: foo.2 $$<
        @echo $^

foo.%:
        @echo -n



Exhibit 1: Make 4.3

$ ls -lah
total 12K
drwxr-xr-x  2 random random 4.0K Sep 28 22:13 .
drwxr-xr-x 17 random random 4.0K Sep 28 21:50 ..
-rw-r--r--  1 random random   73 Sep 28 22:13 Makefile

$ md5sum Makefile
c21dd3f5b109590fa5112b50afc986d6  Makefile

$ make --version
GNU Make 4.3
Built for x86_64-pc-linux-gnu
Copyright (C) 1988-2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

$ make
foo.2 foo.1



Exhibit 2: Make 3.81

$ ls -lah
total 12K
drwxr-xr-x 2 root root 4.0K Sep 28 16:14 .
drwx------ 1 1000 1000 4.0K Sep 28 16:14 ..
-rw-r--r-- 1 root root   73 Sep 28 16:11 Makefile

$ md5sum Makefile
c21dd3f5b109590fa5112b50afc986d6  Makefile

$ make --version
GNU Make 3.81
Copyright (C) 2006  Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.

This program built for x86_64-pc-linux-gnu
$ make
foo.2 foo.1 foo.1

Riccardo P. Bestetti <vmsh0>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by psmith (Posted a comment)
  • -email is unavailable- added by vmsh0 (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follows 1 latest change.

    Date Changed by Updated Field Previous Value => Replaced by
    2022-10-15 psmith Item GroupBug Enhancement

    Back to the top

    Powered by Savane 3.13-d3ae.
    Corresponding source code