Note the FIFO support has been added to the current GCC master.
May I please ask about the next Make release, is it planned any time soon?

I don't see how that patch can work.  There has to be some code somewhere to open the named pipe...??

comment #8:

> This should be resolved in the next release of GNU make which uses named pipes for the jobserver.

Oh, great, thanks a lot for it. We (GCC) have been waiting for the feature for a long time. When can we expect a new make release?

About this statement below:

"This is necessary, because a command can mess up job server operation or a command may expect a specific fd to be available."

a) I wonder how a command can mess up job server operation:

a1) One way would be that a program is looking at all open file descriptors it inherited, and messing with them. I am not sure whether we want to make the job server robust against such attacks.

a2) Another way is that the program has a bug and is not using the job server properly. But I do not think that this scenario is worth much consideration either.


b) If one really wants to stop inheriting the job server and the associated flag, I wonder whether a makefile can filter out the job server options from MAKEFLAGS.

I am actually doing the opposite, I am fetching the jobserver flags from MAKEFLAGS and letting only those through. Check out variable EXTRACT_SELECTED_FLAGS_FROM_MAKEFLAGS in this makefile:

https://github.com/rdiez/JtagDue/blob/master/Toolchain/Makefile

Would it be possible to use GNU Make function $(filter-out pattern...,text) to remove the job server options?


c) I wonder how "a command may expect a specific fd to be available".

A while ago I wanted a way to always pass down the jobserver file descriptors:

Passing jobserver file descriptors to all children
https://lists.gnu.org/archive/html/help-make/2020-02/msg00000.html

I encountered a similar statement then:

"Originally make did pass these file descriptors to all children, but there were bugs filed because some tools invoked by make expected to have specific file descriptors available; having make pass down open FDs caused them to fail."

Can someone provide more detail about which file descriptors a program may expect to be available? Because that is generally considered bad practice, as far as I know.

If this mainly refers to Bash scripts, I found in its manual page that Bash seems to use descriptors above 10 internally, as only the low 10 seem guarantee stability for scripts:

"Redirections using file descriptors greater than 9 should be used with care, as they may conflict with file descriptors the shell uses internally."

Maybe if GNU Make moved those job server descriptors to higher numbers, like Bash does internally, there would be less problems in this area. I cannot imagine that some program or script may assume that file descriptor number 12,345 must be available right now. But maybe I am missing something, I am no expert in this area.

It is not TOO hard to provide a custom value (that doesn't include the jobserver options) for the MAKEFLAGS environment variable we set when forking/execing a command that is not a recursive make.

It's much trickier to reset the MAKEFLAGS make variable to not include those options when expanding the command lines.  For example:

foo:
        +: $(MAKEFLAGS)
        +: $$MAKEFLAGS
        : $(MAKEFLAGS)
        : $$MAKEFLAGS

Ideally when run with -j you'd see the jobserver options in the first two lines and you would NOT see them in the last two lines but this is hard.

Much simpler would be to show the jobserver options in the first three lines but not show them in the last line.  For most uses this is probably sufficient since many programs will be obtaining this information from their environment anyway, not from the command line.  Still, it's an annoying "gotcha".

Regarding named pipes: yes this would be a good option (although it comes with its own issues such as location and permissions); I didn't use this because I wanted the solution to be maximally portable.  But, perhaps there are no useful systems left that both (a) provide standard pipes sufficiently well to support the jobserver and (b) don't provide named pipes.

I fully agree with Stafan that MAKEFLAGS should be aligned with jobserver FDs.
My impression is that a much better approach would be the usage of named pipes instead of file descriptors. I would consider the process relying on fixed file descriptor numbers as a bad practice.

Stefan, the behavior you described is intended.
make closes the pipe unless the command has + or (MAKE) or {MAKE}.
This is necessary, because a command can mess up job server operation or a command may expect a specific fd to be available.