bugmake - Bugs: bug #12229, jobserver information...

 
 

bug #12229: jobserver information communication to indirect make call

Submitted by:  Boris Kolpackov <bosk>
Submitted on:  Sun 06 Mar 2005 12:25:48 PM UTC  
 
Severity: 3 - NormalItem Group: Bug
Status: Not A BugPrivacy: Public
Assigned to: NoneOpen/Closed: Closed
Component Version: 4.0Operating System: Any
Fixed Release: NoneTriage Status: None

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

Sun 26 Jun 2005 03:21:48 AM UTC, comment #2:

I'm closing this as "Not A Bug" for now. Boris, if you have more thoughts about my comments below please add them to the bug report and it can be reopened if necessary.

Paul D. Smith <psmith>
Project Administrator
Thu 09 Jun 2005 05:07:51 PM UTC, comment #1:

The problem with cleaning out MAKEFLAGS is that this message is actually quite USEFUL for people to realize that they have a bug in their makefile, which is causing them to lose their parallelism. The prototypical case of this is using "make" (rather than $(MAKE)) in a script that invokes a sub-make command. 99% of the time when you see this it really IS an error in your makefile, that you should correct if you want to run with -j. So, I kind of like having the warning there.

The second bit, which is a way to invoke a sub-make such that it retains the -n flag but also uses -j, is interesting. I'm not really sure what the purpose of such a thing would be, really: why would you want to run a build with -n AND -j? I guess testing parallelism would be one reason. Are there others?

Anyway, it seems like this could be handled with some fancy footwork to determine if -n was given to the current make, then using it in the command line with sub-makes (prefixed with "+").

Leaving the pipes open is another thought I had, too, but I was (academically) worried that some (badly written) tool that make invoked would expect certain file descriptors to be available, and this would break that. I've never seen a program that badly written, mind you, so it's probably moot. But, as in the first paragraph above, I do think that the error message serves a valuable purpose for people debugging makefiles.

Paul D. Smith <psmith>
Project Administrator
Sun 06 Mar 2005 12:25:48 PM UTC, original submission:

Consider the following makefile

.PHONY: all

ifndef stop

export MAKE

all:
./stop.sh

else

all:
@echo stop

endif

And the stop.sh shell script:

#! /bin/bash

echo MAKE: $MAKE
echo MAKEFLAGS: $MAKEFLAGS

$MAKE --no-print-directory -f makefile stop=1

When executed with -j option it prints the following:

$ make -j 2
./stop.sh
MAKE: make
MAKEFLAGS: --jobserver-fds=3,5 -j
make[1]: warning: jobserver unavailable: using -j1. Add `+' to parent make rule.
stop

Adding '+' in front of ./stop.sh indeed helps but it also has side effects which may be undesirable (e.g., the script could create files, etc).

Some further investigation revieled why the warning is the on the first place. There is a piece of code in job.c around line 1260 that looks like this:

/* If we aren't running a recursive command and we have
a jobserver pipe, close it before exec'ing. */
if (!(flags & COMMANDS_RECURSE) && job_fds[0] >= 0)
{
CLOSE_ON_EXEC (job_fds[0]);
CLOSE_ON_EXEC (job_fds[1]);
}

So what appers to be happening is that the "parent" make closes jobserver's pipes before executing ./stop.sh but
leaves the jobserver-related info in MAKEFLAGS which leads "child" make invocation to belive there is a parent's jobserver. In fact cleaning MAKEFLAGS (using sed) out of --jobserver-fds and -j gets rid of the warning.

It seems to me that the fix should have two parts to it. First, the MAKEFLAGS should be cleaned of any jobserver info when the pipes are closed. Second, it would be nice to have a command prefix like (+) that didn't have any effects other than allowing to use parent's jobserver.

Alternatively, we could simply leave pipes open all the time since it doesn't make much sense to start a separate jobserver from a script.

Boris Kolpackov <bosk>
Project Member

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

CC list is empty

 

Do you think this task is very important?
If so, you can click here to add your encouragement to it.
This task has 0 encouragements so far.

Only logged-in users can vote.

 

Please enter the title of George Orwell's famous dystopian book (it's a date):

 

 

Follow 4 latest changes.

Date Changed By Updated Field Previous Value => Replaced By
Sun 26 Jun 2005 03:21:48 AM UTCpsmithStatusNone=>Not A Bug
  Open/ClosedOpen=>Closed
Sun 06 Mar 2005 06:21:56 PM UTCboskSummaryjobserver information communication to inderect make call=>jobserver information communication to indirect make call
Sun 06 Mar 2005 12:25:48 PM UTCboskCarbon-Copy-=>Added -unavailable-

Back to the top


Powered by Savane 3.1-cleanup1