bugmake - Bugs: bug #44853, gmake: execvp: bash: The parameter...

 
 

bug #44853: gmake: execvp: bash: The parameter or environment lists are too long.

Submitted by:  Daniel Richard G. <iskunk>
Submitted on:  Thu Apr 16 05:37:23 2015  
 
Severity: 3 - NormalItem Group: Bug
Status: NonePrivacy: Public
Assigned to: NoneOpen/Closed: Open
Component Version: 4.1Operating System: POSIX-Based
Fixed Release: NoneTriage Status: None

Add a New Comment (Rich MarkupRich Markup):
   

You are not logged in

Please log in, so followups can be emailed to you.

 

(Jump to the original submission Jump to the original submission)

Tue May 26 15:32:23 2015, comment #8:

SHELL=perl:

.SHELLSOURCE = require "$(.SHELLSCRIPT)";

SHELL=ruby:

.SHELLSOURCE = require "$(.SHELLSCRIPT)"

If SHELL doesn't have an include/source directive, then there's a challenge---but no worse than the E2BIG.

One issue with using a construct that is parallel to .SHELLFLAGS rather than additive is that if you add e.g. -e to .SHELLFLAGS, then the parallel form won't necessarily pick it up.

Daniel Richard G. <iskunk>
Tue May 26 12:06:01 2015, comment #7:

No automated conversion of the command is going to work: you point out csh (which shouldn't be used with make actually) but what if SHELL is Perl? Or Ruby? Or...

If you're going to introduce a new variable anyway, it seems like it would be more straightforward to introduce an alternative to SHELLFLAGS that is used when make decides to create a batch file, rather than a variable containing a "source" operation.

Paul D. Smith <psmith>
Project Administrator
Tue May 26 04:45:37 2015, comment #6:

Well, it would be possible to transform

$(SHELL) -c "very long command"

into

[write "very long command" to /tmp/long.sh]
$(SHELL) -c ". /tmp/long.sh"

so mucking around with .SHELLFLAGS is unnecessary.

The syntax for sourcing a script may vary (e.g. csh would use "source" rather than "."), but a new variable could be provided to control that, like

.SHELLSOURCE = . $(.SHELLSCRIPT)
.SHELLSOURCE = source $(.SHELLSCRIPT)

Would this address your concerns?

Daniel Richard G. <iskunk>
Sat May 23 15:34:11 2015, comment #5:

This is not so easy. Make has exactly one way to invoke an interpreter today: essentially '$(SHELL) $(.SHELLFLAGS) <recipeline>" The setting of .SHELLFLAGS is geared towards this configuration.

Many/most interpreters require a different set of flags to process a file than they do to process a script via the command line, and there is no facility in make today that would allow one to provide those alternative flags. It's not enough to just assume you can remove the "-c" option and call it good. Any solution to this problem would need to provide a solution to this aspect as well.

Paul D. Smith <psmith>
Project Administrator
Fri May 22 21:05:12 2015, comment #4:

I have filed the bug report against Libtool:

http://debbugs.gnu.org/20632

Libtool can handle overlong object lists capably, but the generated Make rule in its current form will not allow it.

Daniel Richard G. <iskunk>
Fri May 22 19:42:05 2015, comment #3:

I'm afraid this is a project using Automake-generated makefiles, and it has to support non-GNU Make programs. If I had full control over the rules, I could certainly go a different route.

(This issue will, in fact, likely require a companion bug to be filed in Libtool. As I found using this hack, even if Make doesn't choke executing that very long command, the interpreter running the libtool script will.)

Daniel Richard G. <iskunk>
Fri May 22 18:47:23 2015, comment #2:

Given that you already have GNU make 4.1, any chance you can use the new $(file ...) function (http://www.gnu.org/software/make/manual/make.html#File-Function) to do the same thing in your own recipe? It could be a path of much less resistance.

David Boyce <boyski>
Fri May 22 15:56:36 2015, comment #1:

I've confirmed that the problem is a too-large argument to "$SHELL -c". The execvp() call is failing with E2BIG.

I've put together a workaround hack for this issue. A patch against git master is attached (which also applies to 4.1).

This adds a handler for E2BIG in exec_command(). The error occurs when executing

argv[0] == $SHELL
argv[1] == "-c"
argv[2] == "way too much shell code"

so it writes argv[2] to a temporary file, and then calls exec_command() recursively to invoke

argv[0] == $SHELL
argv[1] == "/tmp/gmake-e2big.XXXXXX"

This is only a quick-and-dirty hack, however, because (1) the temp file is not cleaned up, as exec_command() does not return, and (2) the code only works in the specific case of a "$SHELL -c '...'" invocation rather than generally.

A real fix may require changes outside of exec_command(), in the places where the exec argument vector is constructed to begin with.

(file #34079)

Daniel Richard G. <iskunk>
Thu Apr 16 05:37:23 2015, original submission:

I am attempting to build a project on AIX 4.3.

The makefile is specifying a library target that has a large number of objects (1000+). GNU Make prints the (very long) command that it is about to execute, but then, it returns

gmake: execvp: bash: The parameter or environment lists are too long.
Makefile:16554: recipe for target 'libfoo.la' failed
gmake: *** [libfoo.la] Error 127

This is with GNU Make 4.1, and GNU Bash 4.3.30.

I suspect that GNU Make is passing the recipe command as a direct argument to "bash -c", which then exceeds the system's limits on execvp(). What is probably needed is to place the command in a temporary file, and then pass that file to the shell for execution.

Daniel Richard G. <iskunk>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

Attached Files
file #34079:  make-e2big-hack.patch added by iskunk (963B - text/x-patch)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by psmith (Posted a comment)
  • -unavailable- added by boyski (Posted a comment)
  • -unavailable- added by iskunk (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follows 1 latest change.

    Date Changed By Updated Field Previous Value => Replaced By
    Fri May 22 15:56:36 2015iskunkAttached File-=>Added make-e2big-hack.patch, #34079

    Back to the top


    Powered by Savane 3.1-cleanup1