helpThe GNU Bourne-Again SHell - Support: sr #110556, aarch64: signal remains blocked...

 
 

sr #110556: aarch64: signal remains blocked after execution of trap handler

Submitted by:  None
Submitted on:  Fri 29 Oct 2021 03:38:56 PM UTC
 
Category:  None Priority:  5 - Normal
Severity:  3 - Normal Status:  Done
Privacy:  Public Assigned to:  None
Originator Email:  -email is unavailable- Open/Closed:  Open
Operating System:  GNU/Linux
* Mandatory Fields

Add a New Comment (Rich Markup)
   

Comment Type & Canned Response:
       

 

( Jump to the original submission )

Fri 19 Nov 2021 09:59:57 PM UTC, comment #7: 

Bash has done it this way for a long time, well before bash-4.3.

It relies on sigsetjmp/siglongjmp to save and restore the signal mask, and the behavior of setjmp/longjmp if HAVE_POSIX_SIGSETJMP is not available.

It never saved and restored the signal mask itself, which was part of the problem with early versions.

The right fix is the one in the devel branch, which assumes that sigsetjmp/siglongjmp are present if it can detect the presence of the POSIX signal functions, instead of defaulting to absent if cross-compiling (you can detect the presence of the signal functions with a simple compile test, but you have to actually run a test to see whether sigsetjmp/siglongjmp restore the signal mask).

Maybe a better test would be to test for sigsetjmp/siglongjmp presence individually if cross-compiling, but I think the current test works.

Chet Ramey <chet>
Project Administrator
Thu 18 Nov 2021 05:56:27 PM UTC, comment #6: 

In the meantime I've done some more testing, and I've got a question about how to best fix this.

I've tested cross-compiling for a 32bit armv7 system, instead of the aarch64 system. It shows the same issue (blocking the signals for longer than expected), so it's not aarch64-specific as I thought at first.

Interestingly, with older versions of bash (such as 4.3.39), the issue does not show up, despite them not detecting POSIX sigsetjmp() either. (i.e. they have /* #undef HAVE_POSIX_SIGSETJMP */ but at least my awkward test case still works as expected)
Bisecting pointed to bash 5.1 (git commit 8868edaf) being the first version that changed that.

While of course I can add a local patch to our local Yocto (the distribution we use for cross-compiling), and perhaps even send that to upstream Yocto, it would be much nicer to have a fix in upstream bash.
I don't know whether that's possible though :)

Is this something that can or should be fixed in bash? I understand some configure checks cannot work when cross-compiling, but perhaps there is another solution to this? On the other hand, the sigsetjmp() check is not the only one that cannot work when cross-compiling, so maybe I'm looking at the wrong thing.

Daniel Klauer <danielklauer>
Wed 03 Nov 2021 08:12:33 PM UTC, comment #5: 

OK, glad to hear that fixed it.

Chet Ramey <chet>
Project Administrator
Tue 02 Nov 2021 03:03:35 PM UTC, comment #4: 

Thanks for checking. I am indeed cross-compiling for the aarch64 machine, and config.h does indeed not #define HAVE_POSIX_SIGSETJMP. (configure said "checking for presence of POSIX-style sigsetjmp/siglongjmp... (cached) missing")
Adding that #define fixes the issue.

Daniel Klauer <danielklauer>
Mon 01 Nov 2021 08:30:57 PM UTC, comment #3: 

I looked at the differences in the strace output. This is the key line present in x86_64 and missing in armv8:

16:58:50.084686 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0

It happens right when trap_handler (the handler for trapped signals) runs sh_longjmp to jump back to the wait builtin to interrupt the wait call.

That jmp_buf is initialized with sigsetjmp (wait_intr_buf, 1), meaning siglongjmp will restore the original signal mask before  calling longjmp. It sure looks like that original signal mask isn't being restored.

This could have a couple of causes. glibc could have a bug in implementing siglongjmp on armv8 that causes it not to restore the signal mask (unlikely). Or libc on armv8 just doesn't implement the signal mask handling in siglongjmp (dunno). Or the configure process could be failing to detect the presence of sigsetjmp/siglongjmp.

If HAVE_POSIX_SIGSETJMP is not defined in config.h, that's the problem.

If you are cross-compiling, that test is going to fail and default to `missing'. You can edit config.cache (if you use it) or define HAVE_POSIX_SIGSETJMP to 1 in config.h and rebuild bash and see if that fixes things.

Chet Ramey <chet>
Project Administrator
Mon 01 Nov 2021 07:29:57 PM UTC, comment #2: 

Before I look at this, this script has several race conditions, not the least of which is that signals are flags, not counters. That is, even if you send several instances of a signal to a process, there is no guarantee that the process will receive more than one. For instance, if the scheduler arranges things so that the `killall' script runs more than once before the first script runs its signal handler, you're going to `lose' signals.

It's also somewhat weird that you're re-instantiating the trap every time through the loop.

I don't have any armv8 systems, but I'll take a look.

Chet Ramey <chet>
Project Administrator
Fri 29 Oct 2021 03:46:21 PM UTC, comment #1: 

Another detail worth mentioning: I've tried not only with SIGUSR1, but also SIGINT - same issue. Didn't test other signals though.

Daniel Klauer <danielklauer>
Fri 29 Oct 2021 03:38:56 PM UTC, original submission:  

Hello,

while using bash 5.1.8 on a 64bit armv8 system, I've encountered an issue with signal handling. The test script sets up a trap for SIGUSR1, then launches a background process (sleep in this case) and uses the "wait" command in a loop to wait for it. If I understood correctly, the "wait" command should return early in case a signal is received, so it can be handled by the trap handler.

When sending signals to this test script while it's running, it appears that only the first signal is received and handled on the aarch64 machine, while on a x86_64 machine it receives and handles all signals.

Test script:

#!/bin/bash
signal=SIGUSR1

on_signal()
{
        echo "trapped $signal"
}
trap on_signal "$signal"

# Start "sleep 5" child process
$(which sleep) 5 &
childpid=$!
echo "started child process"

# Wait for child process to exit, but also handle "wait" returning due to signal.
# "wait" returns immediately with exit code 128+N if signal N is received,
# see section "SIGNALS" in the bash man page.
result=
while [[ -z "$result" || "$result" -gt 128 ]]; do
        trap on_signal "$signal"
        echo "wait"
        wait "$childpid"
        result=$?
        echo "wait returned exit code $result"
done

echo "done"

What I used to send signals to it (run this in parallel to the above test script):

#!/bin/sh
for i in $(seq 1 4); do
        killall -v -SIGUSR1 test.sh
        sleep 1
done

On x86_64, the output is as follows (SIGUSR1 received 4 times, as expected):

started child process
wait
trapped SIGUSR1
wait returned exit code 138
wait
trapped SIGUSR1
wait returned exit code 138
wait
trapped SIGUSR1
wait returned exit code 138
wait
trapped SIGUSR1
wait returned exit code 138
wait
wait returned exit code 0
done

On aarch64, the output is as follows (SIGUSR1 only received once, but should be 4 as on x86_64):

started child process
wait
trapped SIGUSR1
wait returned exit code 138
wait
wait returned exit code 0
done

I've looked at strace output for the bash running the test script, and it seems that on aarch64, it keeps SIGUSR1 blocked after receiving it for the first time:

First wait command, SIGUSR1 is not blocked:

10:25:55.866065 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
10:25:55.866108 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
[...]
10:25:55.866208 wait4(-1, 0xffffdf5e10a0, 0, NULL) = ? ERESTARTSYS (To be restarted if SA_RESTART is set)
10:25:56.627202 --- SIGUSR1 {si_signo=SIGUSR1, si_code=SI_USER, si_pid=532, si_uid=0} ---

Second wait command, SIGUSR1 is/remains blocked:

10:25:56.634380 rt_sigprocmask(SIG_SETMASK, [USR1 CHLD], NULL, 8) = 0
10:25:56.634469 rt_sigprocmask(SIG_BLOCK, [CHLD], [USR1 CHLD], 8) = 0
[...]
10:25:56.634668 wait4(-1, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = 528

On x86_64, that does not happen - SIGUSR1 does not remain blocked for the additional wait commands. I've attached the strace logs.

System information:
- The x86_64 machine uses linux 5.13, gcc 11.2, glibc 2.34 (it's Ubuntu 21.10)
- The aarch64 machine uses linux 5.4, gcc 10.2, glibc 2.33 (it's Yocto hardknott on an embedded system)

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #52168:  test.sh added by None (574B - application/x-shellscript)
file #52169:  strace_aarch64.txt added by None (18KiB - text/plain)
file #52170:  send_signal.sh added by None (77B - application/x-shellscript)
file #52171:  strace_x86_64.txt added by None (28KiB - text/plain)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by chet (Posted a comment)
  • -email is unavailable- added by danielklauer (Posted a comment)
  • -email is unavailable- added by None (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

     

     

     

    Follow 7 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2021-11-03 chet StatusNeed Info => Done
    2021-11-01 chet StatusIn Progress => Need Info
    2021-11-01 chet StatusNone => In Progress
    2021-10-29 None Attached File- => Added test.sh, #52168
        Attached File- => Added strace_aarch64.txt, #52169
        Attached File- => Added send_signal.sh, #52170
        Attached File- => Added strace_x86_64.txt, #52171

    Back to the top


    Powered by Savane 3.9