buggrep - Bugs: bug #17457, "grep -r foo . >...

 
 

You are not allowed to post comments on this tracker with your current authentification level.

bug #17457: "grep -r foo . > somefile" goes into an infinite loop

Submitted by:  None
Submitted on:  Fri 18 Aug 2006 12:08:06 AM UTC  
 
Category: NoneSeverity: 3 - Normal
Item Group: NoneStatus: Invalid
Privacy: PublicAssigned to: None
Open/Closed: Closed

Sat 06 Feb 2010 01:18:36 PM UTC, comment #4:

Hmm, this cannot be fixed. By definition the output file can grow indefinitely, and grep will have to read it until EOF.

Paolo Bonzini <bonzini>
Fri 04 Dec 2009 08:01:19 PM UTC, comment #3:

I didn't test it, but the racy scenario seems plausible. Leaving open since you agree about switching to fts.

Paolo Bonzini <bonzini>
Fri 04 Dec 2009 07:43:12 PM UTC, comment #2:

Hi Paolo,

That script does not evoke an infinite loop for me on F12 on a tmpfs file system.
Does it for you?

Of course, switching to fts is definitely worth doing regardless, since it will remove an entire class of limitations.

Jim Meyering <meyering>
Project Administrator
Fri 04 Dec 2009 07:26:55 PM UTC, comment #1:

Jim, I'm thinking of switching from savedir to fts. Do you think this would fix this? If not, we should close this bug.

Paolo Bonzini <bonzini>
Fri 18 Aug 2006 12:08:06 AM UTC, original submission:

When using grep -r and redirecting output to a file in the traversed directory, grep sometimes enters in an infinite loop, and size of output file grows endelessly.

What happens is : directory is traversed, and every file is searched for a pattern. The line containing the pattern is written to somefile. When it comes to somefile, its content is buffered. Pattern is found in it (because lines containing pattern have already been written). More output is then written to somefile. Then somefile is checked for more content (function fillbuff in grep.c), and newly written lines are parsed. Then more output, then more content, etc.

If output of grep is less than 4096bytes (my system is debian i386) before coming to somefile, and grep is not called with --line-buffered option, that behaviour never occurs. I think that's because data have not been flushed yet. If output is more than 4096, that behaviour doesn't always occur. I couldn't figure out what makes the difference, but I can always reproduce it with that script :

#!/bin/bash

# creates a directory
mkdir tmpdir

cd tmpdir

# creates two files
touch a
touch b

# writes 5000 caracters in another file
# name of that file comes after two other in lexical order
I="0";
while [ $I -lt 5000 ]
do
echo "a" >> c
I=$[$I+1]
done

# recursively grep directory, redirects output
# output of grep must be more than 4096,
grep -r a . > log

One way to solve this problem would be to prevent grep from searching inside a file if that file is associated with stdout. `cat' has that feature. I'll submit a patch based on cat's implementation.

Anonymous

 

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by meyering (Posted a comment)
  • -unavailable- added by bonzini (Posted a comment)
  • -unavailable- added by bonzini
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 3 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Sat 06 Feb 2010 01:18:36 PM UTCbonziniStatusNone=>Invalid
      Open/ClosedOpen=>Closed
    Fri 04 Dec 2009 07:26:55 PM UTCbonziniCarbon-Copy-=>Added meyering

    Back to the top


    Powered by Savane 3.1-cleanup