bugfindutils - Bugs: bug #62622, E2BIG misbehaviour

 
 

bug #62622: E2BIG misbehaviour

Submitter:  Thomas Hurst <freaky>
Submitted:  Mon 13 Jun 2022 01:34:18 PM UTC
   
 
Category:  xargs Severity:  3 - Normal
Item Group:  Wrong result Status:  Confirmed
Privacy:  Public Assigned to:  berny
Originator Name:  Open/Closed:  Open
Release:  4.9.0 Fixed Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Thu 17 Nov 2022 07:12:24 AM UTC, comment #6: 

A reminder about this pending patch.

 Nikolaos Chatzikonstantinou <nchatz314>
Mon 18 Jul 2022 12:20:10 PM UTC, comment #5: 


comment #4:

> Hey all,
>
> I just wanted to show you the conclusion of my investigation of this bug.
>
>
>
>


So here in the attached picture, you can see the variables `state->largest_successful_arg_count, state->smallest_failed_arg_count, limit` that I recorded from a gdb session for `update_limit()` when using the reproducer

    yes | head -n 2000000 > myfile
    gdb --args xargs/xargs -a myfile -s $limit sh -c 'echo hello world'

The green and red dots show the success/fail bounds and the blue dot is the attempt by bc_do_exec().

Every time the blue dot is above the red dot, it goes down because execvp() fails. It goes down by half the difference between the red and green dot.

When it ends up below the red dot, it updates the green dot closer; thus the next time (because we start at the top again), the blue dot takes even longer to reach below the red dot.

I'm continuing to work on this trying to find a way to fix it.

 Nikolaos Chatzikonstantinou <nchatz314>
Mon 18 Jul 2022 12:13:24 PM UTC, comment #4: 

Hey all,

I just wanted to show you the conclusion of my investigation of this bug.




Nikolaos Chatzikonstantinou <nchatz314>
Wed 13 Jul 2022 09:55:04 AM UTC, comment #3: 

I've e-mailed a patch for this. I'm making this comment so that my account does not get deleted (<https://savannah.gnu.org/maintenance/IdleAccounts/>).

 Nikolaos Chatzikonstantinou <nchatz314>
Tue 14 Jun 2022 08:40:31 PM UTC, comment #2: 

Indeed, there seems to be some overflow bug in that heuristic which needs to be fixed.

But a word about the reproducer.

> $ yes | xargs -ts $limit /bin/sh -c 'echo;echo "$(date +%s.%N): $#"' | [...]


If the shell is invoked like that, then one has to pass something as the script name $0 to it;
otherwise the shell would see the first argument as script name:


$ seq 3 | xargs sh -c 'echo "$0: $@"'
1: 2 3

$ seq 3 | xargs sh -c 'echo "$0: $@"' myscript
myscript: 1 2 3


 Bernhard Voelker <berny>
Group administrator
Mon 13 Jun 2022 03:35:24 PM UTC, comment #1: 

I noticed this a while ago but forgot to report it: https://github.com/sharkdp/argmax/issues/1#issuecomment-972035652

`xargs` is trying to find the real limit with binary search: https://git.savannah.gnu.org/cgit/findutils.git/tree/lib/buildcmd.c?id=372cd34894e247fe5c2991eb75185ea2ec850ee2#n190

But `limit` can be far outside the range `[largest_successful_arg_count, smallest_failed_arg_count]`.

 Tavian Barnes <tavianator>
Mon 13 Jun 2022 01:34:18 PM UTC, original submission:  

Attempting to use the argument limit xargs declares in --show-args with -s results in very unusual behaviour from xargs (and presumably other buildcmd users).  This case shows the case of a single byte argument but similar behaviour is observed with longer arguments:

$ limit=$(xargs --show-limits </dev/null 2>&1 | awk '/actually use/{print $NF}')
$ yes | xargs -ts $limit /bin/sh -c 'echo;echo "$(date +%s.%N): $#"' 3>&1 1>&2 2>&3 |
  awk '{printf(".");fflush();}'
.
1655121905.912999462: 130756
.
1655121906.012978619: 130758
....
1655121906.421159119: 204309
.
1655121906.517402529: 57205
............................
1655121909.326479277: 208907
.
1655121909.432364542: 52607

Note the fluctuating argument count and number of command attempts.  This gets quite large, so here is the same output with each line of dots replaced with an attempt count:

1 attempts
1655121905.912999462: 130756
1 attempts
1655121906.012978619: 130758
4 attempts
1655121906.421159119: 204309
1 attempts
1655121906.517402529: 57205
28 attempts
1655121909.326479277: 208907
1 attempts
1655121909.432364542: 52607
409 attempts
1655121951.269149669: 209292
1 attempts
1655121951.364246685: 52222
1684 attempts
1655122117.319284123: 209368
1 attempts
1655122117.424518362: 52146
17382 attempts
1655123835.592837186: 209376
1 attempts
1655123835.690910762: 52138
2 attempts
1655123835.885544564: 130756
1 attempts
1655123835.990007112: 130757
1 attempts
1655123836.088681141: 0
2 attempts
1655123836.290857561: 130756
1 attempts
1655123836.388672514: 130757
1 attempts
1655123836.484069641: 0

It stabilizes at the latter three, going between 130756, 130757, and 0(!) arguments to each invocation repeatedly.

 Thomas Hurst <freaky>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #53451:  Figure_1.png added by nchatz314 (40KiB - image/png - Figure 1)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by nchatz314 (Posted a comment)
  • -email is unavailable- added by berny (Posted a comment)
  • -email is unavailable- added by tavianator (Posted a comment)
  • -email is unavailable- added by freaky (Submitted the item)

    •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 4 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2022-07-18 nchatz314 Attached File- Added Figure_1.png, #53451
    2022-06-14 berny StatusNone Confirmed
        Assigned toNone berny
        ReleaseNone 4.9.0

    Back to the top

    Copyright © 2024 Free Software Foundation, Inc.
    Verbatim copying and distribution of this entire article is permitted in any medium, provided this notice is preserved.
    The Levitating, Meditating, Flute-playing Gnu logo is a GNU GPL'ed image provided by the Nevrax Design Team.
    Page source code

    Powered by Savane 3.14-f13d.
    Corresponding source code