mainThe GNU Bourne-Again SHell - Support: sr #110534, Shebang is ignored when the...

 
 

sr #110534: Shebang is ignored when the application is compiled for the wrong architecture

Submitter:  None
Submitted:  Tue 07 Sep 2021 09:56:33 AM UTC
   
 
Category:  None Priority:  5 - Normal
Severity:  4 - Important Status:  Invalid
Privacy:  Public Assigned to:  None
Originator Email:  -email is unavailable- Open/Closed:  Open
Operating System:  GNU/Linux
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Thu 16 Sep 2021 03:06:53 PM UTC, comment #6: 


> The example file could as well have a shebang with a binary starting at line 2
> (think of some byte-code level interpreter, or some weird self-extracting
> archive) and it would still be classified as "text file" and interpreted by
> Bash.


This suggestion has merit. A file that starts with #! and the name of an invalid interpreter binary that contains binary data on the second and subsequent lines will be treated as a script. I think the best solution for this case is to check the first 128 bytes of the file if it starts with #!, rather than just the first line (where 128 is the maximum number of bytes bash reads from the file when checking).

>
> Anyways.
>
> Let's discuss possible workarounds for this issue instead. My off-the-hat
> ideas are:


You presume that your desired behavior is "correct" and "proper". We don't agree there. POSIX specifies the bash behavior, and nearly all shells behave the same way (see below).

>
> - Would a `set` option or some equivalent that enables the correct behavior
> from within a script be a viable option? It's less discoverable but we already
> teach developers to always use `set -e -o pipefail` and one item more in that
> list wouldn't hurt.


This is a minor issue that occurs extremely rarely. It's not worth a new shell option to do something oddball here.

> - Force-enabling the `HAVE_HASH_BANG_EXEC` compile option is an easy fix,
> although I'd consider it as a "distribution level" solution, which I'd like to
> avoid.


You are certainly welcome to do that.

> - Have a dedicated program to run a script that catches the ENOEXEC and
> handles it properly. Maybe we could integrate it into /usr/bin/env? Either
> way, it would require wrapping all calls with that program so it is not really
> practicable.


This is not appropriate for the shell. If the system is going to have a mechanism that handles ENOEXEC in this fashion, it should be generally available.

> - Switch away from Bash. I still have to check the behavior of more different
> shells, and check if one of them is syntactically sufficiently close to Bash.


Your choices are limited. Given a file with this content on my macOS system:

#! ./linux-binary
echo data 1
echo data 2

Bash, dash, ksh93, yash, gwsh, FreeBSD sh, and NetBSD sh all output

data 1
data 2

The only shells that catch ENOEXEC and print an error message are mksh (though it claims POSIX conformance) and zsh. Even the System V sh behaves like bash.

Chet Ramey <chet>
Group administrator
Thu 09 Sep 2021 07:47:33 PM UTC, comment #5: 

The example file could as well have a shebang with a binary starting at line 2 (think of some byte-code level interpreter, or some weird self-extracting archive) and it would still be classified as "text file" and interpreted by Bash.

Anyways.

Let's discuss possible workarounds for this issue instead. My off-the-hat ideas are:

- Would a `set` option or some equivalent that enables the correct behavior from within a script be a viable option? It's less discoverable but we already teach developers to always use `set -e -o pipefail` and one item more in that list wouldn't hurt.
- Force-enabling the `HAVE_HASH_BANG_EXEC` compile option is an easy fix, although I'd consider it as a "distribution level" solution, which I'd like to avoid.
- Have a dedicated program to run a script that catches the ENOEXEC and handles it properly. Maybe we could integrate it into /usr/bin/env? Either way, it would require wrapping all calls with that program so it is not really practicable.
- Switch away from Bash. I still have to check the behavior of more different shells, and check if one of them is syntactically sufficiently close to Bash.

I'd consider only the first idea as really realistic (w.r.t. to my limited knowledge about that topic). What do you think?

piegames
Thu 09 Sep 2021 03:50:31 PM UTC, comment #4: 

The example file you gave, which is something like

#! /path/to/binary
echo hello

is a text file.

Chet Ramey <chet>
Group administrator
Wed 08 Sep 2021 07:21:31 PM UTC, comment #3: 

The crucial thing is the following sentence:

> If the executable file is not a text file, the shell may bypass this command execution. In this case, it shall write an error message and shall return an exit status of 126.


The key is that "binary file" is under-specified. I think it is valid to treat scripts with a shebang like binary files, since executing them as bash scripts is obviously wrong (if a file has a shebang, it is precisely for the reason that it requires a non-standard interpreter). This is exactly what zsh does, and it indeed resolves the issue.

piegames
Wed 08 Sep 2021 06:43:58 PM UTC, comment #2: 

Bash doesn't interpret #!: that's the kernel/dynamic linker's job. It only pays attention to #! on systems that don't handle it in the kernel, and those are rare these days.

If the execve system call returns ENOEXEC, bash behaves as specified in

https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_09_01_01

and the bash documentation. After checking that the file doesn't appear to be binary, it executes the content as a shell script.

So, in your example, `bash -c ./test' turns into something effectively like `bash ./test'. This all happens in execute_cmd.c:shell_execve().

Chet Ramey <chet>
Group administrator
Tue 07 Sep 2021 12:35:11 PM UTC, comment #1: 

I found the root issue, it's in execute_cmd.h function execute_disk_command: https://git.savannah.gnu.org/cgit/bash.git/tree/execute_cmd.c?id=ce23728687ce9e584333367075c9deef413553fa#n5425

From the documentation above:

>   1) fork ()
>   2) connect pipes
>   3) look up the command
>   4) do redirections
>   5) execve ()
>   6) If the execve failed, see if the file has executable mode set.
>   If so, and it isn't a directory, then execute its contents as
>   a shell script.


Step 6 if the fundamental issue, because it makes the assumption that the content is a shell script [that can be executed with bash]. This assumption breaks when the script has a shebang.

A possible fix would be to ONLY execute the contents as shell script if the file is +x AND it has no shebang set. Otherwise, the operation should fail and the error should be displayed to the user.

For reference, Zsh does quite a few checks on failure before interpreting the file as shell script, including those that check the shebang https://github.com/zsh-users/zsh/blob/master/Src/exec.c#L506

piegames
Tue 07 Sep 2021 09:56:33 AM UTC, original submission:  

Steps to reproduce:

1. Cross-compile yourself an application for a different target. The target doesn't matter, it can even be for embedded/bare metal. (Actually, probably any ELF file that doesn't run on the host system in some way will do fine, but I haven't tested all possibilities yet).
2. Create a script file with that application as shebang and as content `echo hello world`, make it executable.
3. Execute it: `bash -c './test'`. For reference, also execute it with `sh` and `zsh` (sh has the same bug, zsh works fine).

Expected behavior:

An error like `bash: exec format error: /path/to/shebang` is printed and the command has a distinct (non-zero) exit code.

Actual behavior:

Bash ignores the shebang and interprets the script as usual. In this case, it will thus print "hello world". This gets especially funny when the script is for some other programming language.

-----

"Silently failing" is usually a pretty bad thing, but "silently deciding to just make stuff up if things failed" is an order of magnitude worse, thus I've marked this with a higher than normal severity.

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by chet (Posted a comment)
  • -email is unavailable- added by piegames (Posted a comment)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follows 1 latest change.

    Date Changed by Updated Field Previous Value => Replaced by
    2021-09-08 chet StatusNone Invalid

    Back to the top

    Powered by Savane 3.13-f8d8.
    Corresponding source code