bugGNU Parallel - build and execute command lines from standard input in parallel - Bugs: bug #38439, "open files" with...

 
 

bug #38439: "open files" with --files --pipe blocks after a while

Submitted by:  Ole Tange <tange>
Submitted on:  Thu 28 Feb 2013 11:49:39 AM UTC  
 
Category: NoneSeverity: 3 - Normal
Item Group: NoneStatus: Fixed
Privacy: PublicAssigned to: Ole Tange <tange>
Open/Closed: Closed

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

Mon 04 Mar 2013 08:52:47 PM UTC, comment #1:

Fixed in [93dd376].

Ole Tange <tange>
Project AdministratorIn charge of this item.
Thu 28 Feb 2013 11:49:39 AM UTC, original submission:

Hi there ...

I tried to use the --pipe and --files options and found
that parallel sometimes stops working. It seems as if
this is somehow related to ulimit's "open files". I
can reproduce it with the following commands:

$ ulimit -n 500; yes | parallel --files --pipe true | nl | tail -n3
488 /tmp/7T9kEbESkM.par
489 /tmp/7uqQGUS8r9.par
490 /tmp/69W44vvzOv.par

$ ulimit -n 400; yes | parallel --files --pipe true | nl | tail -n3
390 /tmp/5d_P8viQaW.par
391 /tmp/_NNQspv5Yp.par
392 /tmp/e54eVHuqNt.par

$ ulimit -n 300; yes | parallel --files --pipe true | nl | tail -n3
289 /tmp/IUfa7Gzukd.par
290 /tmp/lu9tP4M0nG.par
291 /tmp/dA4yMsrkGV.par

The strange thing is, setting the limit to 200 makes
parallel run fine, but pause for a few seconds every
~185 files.

Ole Tange <tange>
Project AdministratorIn charge of this item.

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by tange (Submitted the item)
  • -unavailable- added by tange
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 5 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Mon 04 Mar 2013 08:52:47 PM UTCtangeStatusConfirmed=>Fixed
      Open/ClosedOpen=>Closed
    Thu 28 Feb 2013 11:49:54 AM UTCtangeStatusNone=>Confirmed
      Assigned toNone=>tange
    Thu 28 Feb 2013 11:49:39 AM UTCtangeCarbon-Copy-=>Added -unavailable-

    Back to the top


    Powered by Savane 3.1-cleanup1