buggrep - Bugs: bug #17457, "grep -r foo . >...


You are not allowed to post comments on this tracker with your current authentication level.

bug #17457: "grep -r foo . > somefile" goes into an infinite loop

Submitted by:  None
Submitted on:  Fri 18 Aug 2006 12:08:06 AM UTC  
Category:  None Severity:  3 - Normal
Item Group:  None Status:  Invalid
Privacy:  Public Assigned to:  None
Open/Closed:  Closed

Sat 06 Feb 2010 01:18:36 PM UTC, comment #4: 

Hmm, this cannot be fixed.  By definition the output file can grow indefinitely, and grep will have to read it until EOF.

Paolo Bonzini <bonzini>
Fri 04 Dec 2009 08:01:19 PM UTC, comment #3: 

I didn't test it, but the racy scenario seems plausible.  Leaving open since you agree about switching to fts.

Paolo Bonzini <bonzini>
Fri 04 Dec 2009 07:43:12 PM UTC, comment #2: 

Hi Paolo,

That script does not evoke an infinite loop for me on F12 on a tmpfs file system.
Does it for you?

Of course, switching to fts is definitely worth doing regardless, since it will remove an entire class of limitations.

Jim Meyering <meyering>
Project Administrator
Fri 04 Dec 2009 07:26:55 PM UTC, comment #1: 

Jim, I'm thinking of switching from savedir to fts.  Do you think this would fix this?  If not, we should close this bug.

Paolo Bonzini <bonzini>
Fri 18 Aug 2006 12:08:06 AM UTC, original submission:  

When using grep -r and redirecting output to a file in the traversed directory, grep sometimes enters in an infinite loop, and size of output file grows endelessly.

What happens is : directory is traversed, and every file is searched for a pattern. The line containing the pattern is written to somefile. When it comes to somefile, its content is buffered. Pattern is found in it (because lines containing pattern have already been written). More output is then written to somefile. Then somefile is checked for more content (function fillbuff in grep.c), and newly written lines are parsed. Then more output, then more content, etc.

If output of grep is less than 4096bytes (my system is debian i386) before coming to somefile,  and grep is not called with --line-buffered option, that behaviour never occurs. I think that's because data have not been flushed yet. If output is more than 4096, that behaviour doesn't always occur. I couldn't figure out what makes the difference, but I can always reproduce it with that script :


# creates a directory
mkdir tmpdir

cd tmpdir

# creates two files
touch a
touch b

# writes 5000 caracters in another file
# name of that file comes after two other in lexical order
while [ $I -lt 5000 ]
    echo "a" >> c

# recursively grep directory,  redirects output
# output of grep must be more than 4096,
grep -r a . > log

One way to solve this problem would be to prevent grep from searching inside a file if that file is associated with stdout. `cat' has that feature. I'll submit a patch based on cat's implementation.



No files currently attached


Depends on the following items: None found

Items that depend on this one: None found


Carbon-Copy List
  • -email is unavailable- added by meyering (Posted a comment)
  • -email is unavailable- added by bonzini (Posted a comment)
  • -email is unavailable- added by bonzini

    Do you think this task is very important?
    If so, you can add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.




    Follow 3 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2010-02-06 bonzini StatusNone => Invalid
        Open/ClosedOpen => Closed
    2009-12-04 bonzini Carbon-Copy- => Added meyering

    Back to the top

    Powered by Savane 3.6