buggrep - Bugs: bug #17457, "grep -r foo . >...

 
 

You are not allowed to post comments on this tracker with your current authentication level.

bug #17457: "grep -r foo . > somefile" goes into an infinite loop

Submitter:  None
Submitted:  Fri 18 Aug 2006 12:08:06 AM UTC
   
 
Category:  None Severity:  3 - Normal
Item Group:  None Status:  Invalid
Privacy:  Public Assigned to:  None
Open/Closed:  Closed

Sat 06 Feb 2010 01:18:36 PM UTC, comment #4: 

Hmm, this cannot be fixed.  By definition the output file can grow indefinitely, and grep will have to read it until EOF.

Paolo Bonzini <bonzini>
Fri 04 Dec 2009 08:01:19 PM UTC, comment #3: 

I didn't test it, but the racy scenario seems plausible.  Leaving open since you agree about switching to fts.

Paolo Bonzini <bonzini>
Fri 04 Dec 2009 07:43:12 PM UTC, comment #2: 

Hi Paolo,

That script does not evoke an infinite loop for me on F12 on a tmpfs file system.
Does it for you?

Of course, switching to fts is definitely worth doing regardless, since it will remove an entire class of limitations.

Jim Meyering <meyering>
Group administrator
Fri 04 Dec 2009 07:26:55 PM UTC, comment #1: 

Jim, I'm thinking of switching from savedir to fts.  Do you think this would fix this?  If not, we should close this bug.

Paolo Bonzini <bonzini>
Fri 18 Aug 2006 12:08:06 AM UTC, original submission:  

When using grep -r and redirecting output to a file in the traversed directory, grep sometimes enters in an infinite loop, and size of output file grows endelessly.

What happens is : directory is traversed, and every file is searched for a pattern. The line containing the pattern is written to somefile. When it comes to somefile, its content is buffered. Pattern is found in it (because lines containing pattern have already been written). More output is then written to somefile. Then somefile is checked for more content (function fillbuff in grep.c), and newly written lines are parsed. Then more output, then more content, etc.

If output of grep is less than 4096bytes (my system is debian i386) before coming to somefile,  and grep is not called with --line-buffered option, that behaviour never occurs. I think that's because data have not been flushed yet. If output is more than 4096, that behaviour doesn't always occur. I couldn't figure out what makes the difference, but I can always reproduce it with that script :

#!/bin/bash

# creates a directory
mkdir tmpdir

cd tmpdir

# creates two files
touch a
touch b

# writes 5000 caracters in another file
# name of that file comes after two other in lexical order
I="0";
while [ $I -lt 5000 ]
do
    echo "a" >> c
    I=$[$I+1]
done

# recursively grep directory,  redirects output
# output of grep must be more than 4096,
grep -r a . > log




One way to solve this problem would be to prevent grep from searching inside a file if that file is associated with stdout. `cat' has that feature. I'll submit a patch based on cat's implementation.

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by meyering (Posted a comment)
  • -email is unavailable- added by bonzini (Posted a comment)
  • -email is unavailable- added by bonzini
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

     

    Follow 3 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2010-02-06 bonzini StatusNone Invalid
        Open/ClosedOpen Closed
    2009-12-04 bonzini Carbon-Copy- Added meyering

    Back to the top

    Powered by Savane 3.13-4448.
    Corresponding source code