bugmake - Bugs: bug #53201, Target runs incorrect command when...


bug #53201: Target runs incorrect command when shebang line exceeds kernel limit

Submitted by:  Casey McGinty <cmcginty>
Submitted on:  Wed 21 Feb 2018 07:35:13 AM UTC  
Severity: 3 - NormalItem Group: Bug
Status: NonePrivacy: Public
Assigned to: NoneOpen/Closed: Open
Component Version: 3.81Operating System: POSIX-Based
Fixed Release: NoneTriage Status: None

Add a New Comment(Rich Markup)

You are not logged in

Please log in, so followups can be emailed to you.


Thu 22 Feb 2018 01:12:28 PM UTC, comment #3:

Sorry, I was thinking of a different kernel limit but my second observation, that the difference observed has to do with fast path optimization, remains. Working around it by setting SHELL=//bin/sh in the makefile is probably safer than .ONESHELL. Again, discussion relating to the fix per se belongs to Paul.

David Boyce <boyski>
Thu 22 Feb 2018 07:06:23 AM UTC, comment #2:

See http://man7.org/linux/man-pages/man2/execve.2.html

A maximum line length of 127 characters is allowed for the first line in an interpreter script.

My kernel is Linux 3.10.0-693.2.2.el7.x86_64 #1 SMP Tue Sep 12 22:26:13 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

This is a production system and can not be updated, however I don't see how that would help.

The kernel define is still set to 127:

If you think this is incorrect, please explain where my misunderstanding is.

Casey McGinty <cmcginty>
Wed 21 Feb 2018 01:06:40 PM UTC, comment #1:

I think a couple of facts here may need correcting though the issue is quite real. I suspect you are not running into a kernel limit since modern Linux kernels have no such limit (at one time the limit was 131K, then doubled to 262K, then removed altogether). However, the mere fact that the kernel imposes no limit doesn't mean individual utilities, in particular the shell, have the same property.

Most likely the limit being exceeded belongs to bash. The so-called fast-path optimization in make skips the shell and execs the recipe directly if it contains no shell special characters, thus in your original test case the shell isn't used which explains the variance. Use of .ONESHELL suppresses fast-path causing the shell to come back in and fail properly.

There are other ways of suppressing fast path to test this thesis; try appending a semicolon to the recipe line or overriding the SHELL variable. One of my favorite tricks is SHELL=//bin/sh which addresses the same executable while convincing make it's different, thus disabling fast path.

I have no wisdom to offer regarding the fix. Your solution sounds fine but that belongs to Paul.

David Boyce <boyski>
Wed 21 Feb 2018 07:35:13 AM UTC, original submission:

I have a Python script (in ./env/bin/pip) generated by virtualenv with a very long shebang line. The virtualenv is active and the PATH env contains "./env/bin;...." before all other paths. When I run "pip" from the system shell I see the following error:

... /bin/python: can't open file 'pip': [Errno 2] No such file or directory

This error is expected because the shebang line is beyond the limit allowed by the kernel.

However when the same command runs in Make, it does NOT fail. Instead, it succeeds by running /bin/pip. This is very unusual since the local script `./env/bin/pip` is first in the PATH . The problem with running the system pip is that this should never happen as the local environment thinks it is inside of Python virtualenv. By running the system pip, we are tainting the system Python runtime environment with unwanted packages.

Some debugging with strace revealed exactly what Make is doing:

[pid 8578] execve("/var/lib/jenkins/workspace/..../env/bin/pip", ["pip", "list"], [/* 26 vars */]) = -1 ENOENT (No such file or directory)
[pid 8578] execve("/sbin/pip", ["pip", "list"], [/* 26 vars */]) = -1 ENOENT (No such file or directory)
[pid 8578] execve("/bin/pip", ["pip", "list"], [/* 26 vars */] <unfinished ...>

Note that adding the .ONESHELL directive resolve the issue and causes Make returns the expected error string above.

I think the proper fix is to stat() the file before accepting the ENOENT error from execve. If the file exists and execve still returns an ENOENT error, Make should fail the target and return the error to the user.

Casey McGinty <cmcginty>


(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):

No files currently attached


Depends on the following items: None found

Items that depend on this one: None found


Carbon-Copy List
  • -unavailable- added by boyski (Posted a comment)
  • -unavailable- added by cmcginty (Submitted the item)

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.


    Please enter the title of George Orwell's famous dystopian book (it's a date):



    No Changes Have Been Made to This Item

    Back to the top

    Powered by Savane 3.1-cleanup1