bugfindutils - Bugs: bug #48030, -exec + does not pass all...

 
 

bug #48030: -exec + does not pass all arguments for certain specific filename lengths

Submitted by:  None
Submitted on:  Thu 26 May 2016 04:07:06 PM UTC  
Votes:  1  
 
Category:  find Severity:  4 - Important
Item Group:  Wrong result Status:  Fixed
Privacy:  Public Assigned to:  Bernhard Voelker <berny>
Originator Name:  Joe Philip Ninan Originator Email:  -email is unavailable-
Open/Closed:  Open Release:  4.4.2
Fixed Release:  None

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

( Jump to the original submission)

Sun 05 Jun 2016 03:34:20 PM UTC, comment #9:

The patch looks good to me. Thanks for taking care of this.

James Youngman <jay>
Project Administrator
Tue 31 May 2016 08:45:32 AM UTC, comment #8:

Thanks for the review - pushed with that fixed (and a little tweak in the commit message):

http://git.sv.gnu.org/cgit/findutils.git/commit/?id=8cdc9767e3

Have a nice day,
Berny

Bernhard Voelker <berny>
Project AdministratorIn charge of this item.
Mon 30 May 2016 03:18:50 PM UTC, comment #7:

Looks good to me. I can confirm it fixes the problem. As a minor nit, there are two trailing white-spaces in the test case.

Kamil Dudka <kdudka>
Mon 30 May 2016 12:47:09 PM UTC, comment #6:

Please review the attached patch.

Bernhard Voelker <berny>
Project AdministratorIn charge of this item.
Mon 30 May 2016 12:27:50 AM UTC, comment #5:

This issue seems to happen when the 128k buffer for the
multiple execution is full and therefore executed, and if
then only one additional entry has to be processed.

The cause is in bc_push_arg() where state->todo is reset
to 0 during such immediate execution; i.e., setting it to 1
is done too early in that function.

As the cause of this bug is in the common lib/buildcmd.c source,
xargs(1) may also be affected, but I couldn't trigger the bug
there so far.

Providing a patch with a test case soon.

Bernhard Voelker <berny>
Project AdministratorIn charge of this item.
Sat 28 May 2016 11:38:37 AM UTC, comment #4:

This also applies to oldfind from 4.4.2.

Andreas Metzler <ametzler>
Fri 27 May 2016 12:53:21 PM UTC, comment #3:

I could reproduce this bug on latest Fedora which contains version findutils 4.5.14

Joe Philip Ninan <indiajoe>
Fri 27 May 2016 12:37:27 PM UTC, comment #2:

Is this bug reproducible with the current stable release (4.6.0)?

Anonymous
Thu 26 May 2016 08:10:25 PM UTC, comment #1:

Just to make sure this bug has nothing to do with any of the individual filenames, I tried it on generic filenames in such a way the total length of the arguments is same.

# Following is how one can simply reproduce the bug

## First generate a very specific size sample directory
### Length of the filenames are crucial here

$ mkdir RashuBug
$ for i in $(seq -f "%04g" 901) ; do touch RashuBug/abcdefghijklmnopqrstuv$i ; done
$ for i in $(seq -f "%04g" 902 3719) ; do touch RashuBug/abcdefghijklmnopqrstu$i ; done

$ ls RashuBug/* | wc -l
3719 # Number of files we have created in directory

## Now create an echo command which is 6 characters long
$ ln -s /bin/echo /bin/echooo # as root

## See the difference in the output of the following two commands

$ find RashuBug -type f -exec echo {} + | tr ' ' '\n' | wc -l
3719 # Correct answer

$ find RashuBug -type f -exec echooo {} + | tr ' ' '\n' | wc -l
3718 # Wrong answer (Last file went missing!)

Joe Philip Ninan <indiajoe>
Thu 26 May 2016 04:07:06 PM UTC, original submission:

# 1. the version of findutils you are using

We could reproduce this bug over many machines. All of them had find (GNU findutils) 4.4.2

# 2. what you were trying to do

We were trying to execute a command using find's -exec comand +

# 3. the exact command line that you used

find RashuBug -type f -exec md5sum {} + | wc -l

# 4. what you expected to happen

This directory RashuBug has 3719 files in it. So we expected 3719 lines of output

# 5. precisely what did happen.

The last file was not passed to md5sum command, and we got only 3718 lines as output.

# 6. How to reproduce this bug

I am attaching RashuBug.tar.gz file.
Extract this tar.gz file to obtain a directory RashuBug/ containing 3719 empty files.

give the following command,
$ find RashuBug -type f -exec md5sum {} + | wc -l

You will get output 3718 (Because last file missing!)

$ cp -r RashuBug RashuBugs
$ find RashuBugs -type f -exec md5sum {} + | wc -l

You will get all 3719 output this time.

# 7. Interesting Observations which might provide some clues.

As metione above, this bug disappears if the length of the directory name is not exactly 8. For example if you rename this Directory as RashuBugs, it will give all 3719 output.

This bug also disappears if the command, in this case md5sum is not exactly 6 characters.

For example, the following command works perfectly
$ find RashuBug -type f -exec echo {} + | tr ' ' '\n' | wc -l

But if you create a new command 6 characters long

$ ln -s /usr/bin/echo /usr/bin/echooo
$ find RashuBug -type f -exec echooo {} + | tr ' ' '\n' | wc -l

Then the bug re-appears, and it is missing the last file.

Hence the issue is not with md5sum command. For this very specific length of filenames, directory name and command name, find command is dropping the last file it finds while passing as arguments to the -exec command.

If you delete any one file from the the RashuBug directory, also this bug disappears.
But if you add a new file by
$ touch RashuBug/new.fits

Then the bug appears when the command it is excuting is 4 characters long.
So this bug is appearing for this specif length of filenames.

We discovered this bug, by accident, while dealing with large collection of telescope observatory data.

The contents of the files are not important to reproduce this bug. Hence the attached RashuBug.tar.gz contains empty files with exact filenames of our data created using touch command!

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #37280:  RashuBug.tar.gz added by None (44KiB - application/x-gzip - Filenames which can reproduce this bug)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by jay (Posted a comment)
  • -email is unavailable- added by kdudka (Posted a comment)
  • -email is unavailable- added by berny (Posted a comment)
  • -email is unavailable- added by indiajoe (Voted in favor of this item)
  • -email is unavailable- added by indiajoe (Posted a comment)
  • -email is unavailable- added by None (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can add your encouragement to it.
    This task has 1 encouragement so far.

    Only logged-in users can vote.

     

     

     

    Follow 8 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2016-05-31 berny StatusIn Progress => Fixed
    2016-05-30 berny Attached File- => Added 0001-Fix-bug-48030-find-exec-does-not-pass-all-arguments-.patch, #37304
        Summary-exec + does not pass all arguments for ceratin specific filename lengths => -exec + does not pass all arguments for certain specific filename lengths
    2016-05-30 berny Severity3 - Normal => 4 - Important
        StatusNone => In Progress
        Assigned toNone => berny
    2016-05-27 indiajoe Carbon-Copy- => Added indiajoe
    2016-05-26 None Attached File- => Added RashuBug.tar.gz, #37280

    Back to the top


    Powered by Savane 3.3