bugfindutils - Bugs: bug #64451, Unexpected behaviour of xargs when...

 
 

bug #64451: Unexpected behaviour of xargs when multiple children exit with 255 in parallel

Submitter:  None
Submitted:  Thu 20 Jul 2023 07:46:06 AM UTC
Votes: 1
 
Category:  xargs Severity:  3 - Normal
Item Group:  Wrong result Status:  None
Privacy:  Public Assigned to:  None
Originator Name:  Originator Email:  -email is unavailable-
Open/Closed:  Open Release:  4.9.0
Fixed Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Thu 20 Jul 2023 06:02:30 PM UTC, comment #5: 

From wait(2):
WIFSTOPPED(*_wstatus_)*
    returns true if the child process was stopped by delivery of a signal; this is possible only if the call was done using WUNTRACED or when the child is being traced (see ptrace(2)).

    OK so I guess this functionality is not supposed to be used as was testing it.

Anonymous
Thu 20 Jul 2023 05:53:17 PM UTC, comment #4: 

I still can't figure out how catching SIGSTOP is supposed to work. Neither SIGSTOP nor SIGTSTP seem to be caught.

Anonymous
Thu 20 Jul 2023 05:44:11 PM UTC, comment #3: 

I've been able to fix this behaviour by explicitly calling wait_for_proc_all(). I can't imagine any reprocussions from doing this, but the current code expects wait_for_proc_all() to be called atexit() so I would need some insight into whether this is good or bad.

(file #54946)

Anonymous
Thu 20 Jul 2023 03:23:25 PM UTC, comment #2: 

    Because checks for WSTOPSIG and WTERMSIG are done in the same place, sending SIGTERM or SIGSTOP to the children should also cause the same behaviour. In reality though trying to kill chuldren with this tester

seq 3 | xargs -n1 -P0 sh -c 'for i do echo start $i; sleep 20; echo stop $i; done' _; echo xargs exited here with $?

I discovered that kill pid causes this behaviour.

start 1
start 2
start 3
xargs: sh: terminated by signal 15 # I run "kill pid" on another terminal
xargs exited here with 125
stop 1
stop 3

But kill -STOP pid causes this:

start 1
start 2
start 3
# I run "kill -STOP pid" on another terminal
stop 1
stop 3
# xargs now just waits. I run "kill -CONT pid" on another terminal.
stop 2
xargs exited here with 0


Anonymous
Thu 20 Jul 2023 08:03:33 AM UTC, comment #1: 

    A workaround for scripts is to perform checks as soon as possible and exit 1 (or other value) instead. Potentially hundreds of processes will still be spawned but this avoids potentially processing outputs that are not yet complete.
    Another way is to catch 124 and interrupt the script right there.

Anonymous
Thu 20 Jul 2023 07:46:06 AM UTC, original submission:  

Synopsis

    When xargs is invoked with -P option set to a value other than 1 or 2 and more that one child exits with 255, xargs fails to wait on all children before exiting.
    -P0 breaks in the same manner on just one exit 255.
    The case for -P2 is a degenerate case where the second child exiting would cause this bug, but as it is the last running child, exiting with 255 does not cause any misbehaviour.

Examples


# One exit 255
seq 10 | xargs -n1 -P4 sh -c 'for i do echo start $i; sleep $i; if [ $i -eq 3 ]; then echo exit $i; exit 255; fi; echo stop $i; done' _; echo xargs exited here with $?
start 1
start 2
start 3
start 4
stop 1
start 5
stop 2
start 6
exit 3
xargs: sh: exited with status 255; aborting
stop 4
stop 5
stop 6
xargs exited here with 124

    After job 3 exited with 255, xargs correctly waits for already started jobs to finish.

# Two exit 255s
seq 10 | xargs -n1 -P4 sh -c 'for i do echo start $i; sleep $i; if [ $i -ge 3 ] && [ $i -le 4 ]; then echo exit $i; exit 255; fi; echo stop $i; done' _; echo xargs exited here with $?
start 1
start 2
start 3
start 4
stop 1
start 5
stop 2
start 6
exit 3
xargs: sh: exited with status 255; aborting
exit 4
xargs: sh: exited with status 255; aborting
xargs exited here with 124
stop 5
stop 6

    xargs exits after job 4. Jobs 5 and 6 continue to run in background. It can no longer be guaranteed that jobs 5 and 6 will complete before further processing in some script.

POSIX compliance

    This breaks a POSIX requirement.
From xargs(1p):
    Implementations wanting to provide parallel operation of the invoked utilities are encouraged to add an option enabling parallel invocation, but should still wait for termination of all of the children before xargs terminates normally.

Quick debugging

    After adding unconditional error messages to the wait_for_proc_all() function

diff --git a/xargs/xargs.c b/xargs/xargs.c
index fdede10..9bc7aa7 100644
--- a/xargs/xargs.c
+++ b/xargs/xargs.c
@@ -1597,6 +1597,7 @@ wait_for_proc (bool all, unsigned int minreap)
 static void
 wait_for_proc_all (void)
 {
+  error (0, 0, _("wait_for_proc_all enter"));
   static bool waiting = false;

   /* This function was registered by the original, parent, process.
@@ -1614,6 +1615,7 @@ wait_for_proc_all (void)
   wait_for_proc (true, 0u);
   waiting = false;

+  error (0, 0, _("wait_for_proc_all before child_error test"));
   if (original_exit_value != child_error)
     {
       /* wait_for_proc () changed the value of child_error ().  This

and testing xargs again, the following is observed:

# One exit 255
seq 10 | ./xargs -n1 -P4 sh -c 'for i do echo start $i; sleep $i; if [ $i -ge 3 ] && [ $i -le 3 ]; then echo exit $i; exit 255; fi; echo stop $i; done' _; echo xargs exited here with $?
start 1
start 2
start 3
start 4
stop 1
start 5
stop 2
start 6
exit 3
./xargs: sh: exited with status 255; aborting
./xargs: wait_for_proc_all enter
stop 4
stop 5
stop 6
./xargs: wait_for_proc_all before child_error test
xargs exited here with 124


# Two exit 255s
seq 10 | ./xargs -n1 -P4 sh -c 'for i do echo start $i; sleep $i; if [ $i -ge 3 ] && [ $i -le 4 ]; then echo exit $i; exit 255; fi; echo stop $i; done' _; echo xargs exited here with $?
start 1
start 2
start 3
start 4
stop 1
start 5
stop 2
start 6
exit 3
./xargs: sh: exited with status 255; aborting
./xargs: wait_for_proc_all enter
exit 4
./xargs: sh: exited with status 255; aborting
xargs exited here with 124
stop 5
stop 6

    It can be seen that in the second case wait_for_proc_all() did not reach before child_error test line.

My hypothesis

    As wait_for_proc_all() is registered via atexit() it is called when the first exit 255 is handled with error() (which itself calls exit()), then wait_for_proc_all() calls wait_for_proc() inside of which another exit() is called.
According to atexit(3):
    POSIX.1 says that the result of calling exit(3) more than once (i.e., calling exit(3) within a function registered using atexit()) is undefined. On some systems (but not Linux), this can result in an infinite recursion; portable programs should not invoke exit(3) inside a function registered using atexit().

In the case of -P0

    If the same command line is run with -P0 something else happens:

# One exit 255 is now enough to exit xargs
seq 10 | ./xargs -n1 -P0 sh -c 'for i do echo start $i; sleep $i; if [ $i -ge 3 ] && [ $i -le 4 ]; then echo exit $i; exit 255; fi; echo stop $i; done' _; echo xargs exited here with $?
start 1
start 2
start 3
start 4
start 5
start 7
start 6
start 8
start 9
./xargs: wait_for_proc_all enter
start 10
stop 1
stop 2
exit 3
./xargs: sh: exited with status 255; aborting
xargs exited here with 124
exit 4
stop 5
stop 6
stop 7
stop 8
stop 9
stop 10

    As xargs has read in all the input it enters wait_for_proc_all(), now a single exit 255 is enough to exit xargs.

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #54946:  0001-Add-quick-and-dirty-fix-for-bug-64451.patch added by None (2KiB - text/x-patch - I don't use email so I don't know if the From: field is valid or not.)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by gabrielravier (Voted in favor of this item)
  •  

    There is 1 vote so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 2 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2024-02-21 gabrielravier Carbon-Copy- Added gabrielravier
    2023-07-20 None Attached File- Added 0001-Add-quick-and-dirty-fix-for-bug-64451.patch, #54946

    Back to the top

    Powered by Savane 3.13-02a9.
    Corresponding source code