bugmake - Bugs: bug #63362, make 4.4: different behavior with...

 
 

bug #63362: make 4.4: different behavior with -j1 and -j2 when building manpages from git

Submitter:  Alexander Kanavin <kanavin>
Submitted:  Mon 14 Nov 2022 10:04:36 AM UTC
   
 
Severity:  3 - Normal Item Group:  Bug
Status:  Wont Fix Privacy:  Public
Assigned to:  None Open/Closed:  Closed
Component Version:  4.4 Operating System:  POSIX-Based
Fixed Release:  None Triage Status:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Sun 04 Feb 2024 11:44:10 PM UTC, comment #9: 

I haven't seen a groundswell of other makefiles with similar issues and the issue with the Git makefiles has been addressed, so I won't try to do something about this.  Please add a comment if you disagree and we can discuss.

Paul D. Smith <psmith>
Group administrator
Tue 29 Nov 2022 09:43:15 AM UTC, comment #8: 

Thanks, I checked locally that this solves the problem.

Alexander Kanavin <kanavin>
Mon 28 Nov 2022 10:26:18 PM UTC, comment #7: 

I posted a patch for the Git build system to the Git mailing list, to avoid this issue; that change will be needed because the behavior of the Git build system isn't really correct, although prior to 4.4 it was basically ignored.

In GNU make 4.4 that behavior is deprecated, and in the next non-patch release it will be illegal.

I'm still trying to figure out whether to try to do something about this in the short term, in GNU make.  It will depend on how tricky the solution is.  It might just be a a missing implicit rule check.

I discovered why you didn't get the warning about missing peer targets; if the target is not built we don't bother checking this.  That seems OK to me.

Paul D. Smith <psmith>
Group administrator
Wed 16 Nov 2022 02:44:44 PM UTC, comment #6: 

I was able to reproduce this incorrect behavior but haven't had time to look at it closely yet.  It's definitely extremely odd.

Paul D. Smith <psmith>
Group administrator
Tue 15 Nov 2022 02:59:27 PM UTC, comment #5: 

Wow that is really confusing because the output shows that those prerequisites are already up to date.  I'm going to have to see if I can reproduce this here.

Paul D. Smith <psmith>
Group administrator
Tue 15 Nov 2022 09:31:01 AM UTC, comment #4: 

I've attached the two outputs from:

$ ../../make-4.4/make -j1 -d git-add.1 > j1-with-d

$ ../../make-4.4/make -j2 -d git-add.1 > j2-with-d

Alexander Kanavin <kanavin>
Mon 14 Nov 2022 08:38:57 PM UTC, comment #3: 

Hm.  Unfortunately that didn't help me much.  I don't see why make thinks all those files have been updated.

Can you please run with the full -d and attach the generated output as a file to this issue?  Possibly even compressed if it's too large: it's going to be too big to include verbatim in a comment.

Paul D. Smith <psmith>
Group administrator
Mon 14 Nov 2022 01:24:51 PM UTC, comment #2: 

Here's two consecutive invocations with -j1 and then with -j2, with --trace added:


alex@alex-lx-laptop:~/development/git/Documentation$ ../../make-4.4/make -j1 --trace git-add.1
    SUBDIR ../
make[1]: 'GIT-VERSION-FILE' is up to date.
Makefile:327: update target 'GIT-ASCIIDOCFLAGS' due to: FORCE
FLAGS='asciidoc  -f asciidoc.conf -amanversion=2.38.1.420.g319605f8f0 -amanmanual='\''Git Manual'\'' -amansource='\''Git'\'':xhtml11:docbook'; \
    if test x"$FLAGS" != x"`cat GIT-ASCIIDOCFLAGS 2>/dev/null`" ; then \
        echo >&2 "    * new asciidoc flags"; \
        echo "$FLAGS" >GIT-ASCIIDOCFLAGS; \
            fi
alex@alex-lx-laptop:~/development/git/Documentation$ ../../make-4.4/make -j2 --trace git-add.1
    SUBDIR ../
make[1]: 'GIT-VERSION-FILE' is up to date.
Makefile:327: update target 'GIT-ASCIIDOCFLAGS' due to: FORCE
FLAGS='asciidoc  -f asciidoc.conf -amanversion=2.38.1.420.g319605f8f0 -amanmanual='\''Git Manual'\'' -amansource='\''Git'\'':xhtml11:docbook'; \
    if test x"$FLAGS" != x"`cat GIT-ASCIIDOCFLAGS 2>/dev/null`" ; then \
        echo >&2 "    * new asciidoc flags"; \
        echo "$FLAGS" >GIT-ASCIIDOCFLAGS; \
            fi
Makefile:355: update target 'git-add.1' due to: git-add.xml manpage-base-url.xsl manpage-bold-literal.xsl manpage-normal.xsl manpage-quote-apos.xsl manpage.xsl
echo '   ' XMLTO git-add.1;xmlto -m manpage-normal.xsl -m manpage-bold-literal.xsl -m manpage-base-url.xsl man git-add.xml
    XMLTO git-add.1
Makefile:355: warning: pattern recipe did not update peer target 'git-add.7'.
Makefile:355: warning: pattern recipe did not update peer target 'git-add.5'.


Alexander Kanavin <kanavin>
Mon 14 Nov 2022 01:15:54 PM UTC, comment #1: 

I am not sure why things behave differently between -j1 and -j2 I will have to investigate it.  You can more easily determine what is going on if you invoke make with the `--trace` option.

However, this rule seems wrong and always has been:

%.1 %.5 %.7 : %.xml manpage-base-url.xsl $(wildcard manpage*.xsl)
        $(QUIET_XMLTO)$(XMLTO) -m $(MANPAGE_XSL) $(XMLTO_EXTRA) man $<

To make this means that one invocation of that recipe for a file xyz.xml will build ALL the targets xyz.1, xyz.5, and xyz.7.  Which, it does not do as best as I can tell.

Possibly the author of the makefile thought that the above command is equivalent to writing this:

%.1 : %.xml manpage-base-url.xsl $(wildcard manpage*.xsl)
        $(QUIET_XMLTO)$(XMLTO) -m $(MANPAGE_XSL) $(XMLTO_EXTRA) man $<
%.5 : %.xml manpage-base-url.xsl $(wildcard manpage*.xsl)
        $(QUIET_XMLTO)$(XMLTO) -m $(MANPAGE_XSL) $(XMLTO_EXTRA) man $<
%.7 : %.xml manpage-base-url.xsl $(wildcard manpage*.xsl)
        $(QUIET_XMLTO)$(XMLTO) -m $(MANPAGE_XSL) $(XMLTO_EXTRA) man $<

but it is not the same thing; if you want three different rules you have to, unfortunately, write them all out like that.  This is why you're seeing that warning.  But I don't know that this has anything to do with why things are being rebuilt.

Paul D. Smith <psmith>
Group administrator
Mon 14 Nov 2022 10:04:36 AM UTC, original submission:  

When building git v2.38.1
(https://github.com/git/git/tags)

I noticed that make 4.4 behaves differently depending on parallelism settings.


alex@alex-lx-laptop:~/development/git/Documentation$ ../../make-4.4/make -j1 git-add.1
    SUBDIR ../
make[1]: 'GIT-VERSION-FILE' is up to date.

vs

alex@alex-lx-laptop:~/development/git/Documentation$ ../../make-4.4/make -j2 git-add.1
    SUBDIR ../
make[1]: 'GIT-VERSION-FILE' is up to date.
    XMLTO git-add.1
Makefile:355: warning: pattern recipe did not update peer target 'git-add.7'.
Makefile:355: warning: pattern recipe did not update peer target 'git-add.5'.


Manpages are rebuilt without need, if j is more than one. I believe this is triggered by FORCE flag for one of the rules in the makefile, but still the difference in behavior depending on what is in -j does not seem correct.
https://github.com/git/git/blob/v2.38.1/Documentation/Makefile#L326

I also confirmed that with make 4.3 this does not happen: manpages are not rebuilt, regardless of -j. FORCE does not have an effect on this.

Alexander Kanavin <kanavin>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #53956:  j1-with-d added by kanavin (157KiB - application/octet-stream - make 4.4 output with -d)
file #53957:  j2-with-d added by kanavin (159KiB - application/octet-stream - make 4.4 output with -d)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by psmith (Posted a comment)
  • -email is unavailable- added by kanavin (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 4 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2024-02-04 psmith StatusNone Wont Fix
        Open/ClosedOpen Closed
    2022-11-15 kanavin Attached File- Added j1-with-d, #53956
        Attached File- Added j2-with-d, #53957

    Back to the top

    Powered by Savane 3.12.
    Corresponding source code