bugGNU Parallel - build and execute command lines from standard input in parallel - Bugs: bug #39554, Feature request: line buffered...

 
 

bug #39554: Feature request: line buffered output

Submitted by:  Ole Tange <tange>
Submitted on:  Tue 23 Jul 2013 09:50:59 AM UTC  
 
Category: NoneSeverity: 1 - Wish
Item Group: NoneStatus: Fixed
Privacy: PublicAssigned to: Ole Tange <tange>
Open/Closed: Closed

Add a New Comment (Rich MarkupRich Markup):
   

You are not logged in

Please log in, so followups can be emailed to you.

 

Tue 30 Jul 2013 09:17:03 PM UTC, comment #2:

Fixed in [5b16a48] and [df204a6].

Ole Tange <tange>
Project AdministratorIn charge of this item.
Thu 25 Jul 2013 02:05:18 PM UTC, comment #1:

If non blocking read does not work on old systems try:

$pos_now = tell $in_fd;
# Seek to the end
seek $in_fd, 0, 1;
$pos_end = tell $in_fd;
seek $in_fd, $pos_now, 0;
if($pos_now != $pos_end) {
read $pos_end - $pos_now;
$pos_now = $pos_end;
}

Ole Tange <tange>
Project AdministratorIn charge of this item.
Tue 23 Jul 2013 09:50:59 AM UTC, original submission:

For some jobs it will be nice to get output from the jobs while the jobs are running.

-u will do that but may mix up the lines.

So what is wanted is a non-mixing way to get full lines from each system.

The idea being that instead of calling print when a child dies, the file handles are read non-blocking periodically:

for $fd {
$buf .= read_non_block($fd,1_000_000);
$last_newline = index $buf,"\n",BACKWARDS;
if $last_newline:
print substr($buf,$last_newline);
substr($buf,$last_newline)="";
}
}

When the job dies: Read the rest and print it.

Ole Tange <tange>
Project AdministratorIn charge of this item.

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by tange (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 2 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Tue 30 Jul 2013 09:17:03 PM UTCtangeStatusConfirmed=>Fixed
      Open/ClosedOpen=>Closed

    Back to the top


    Powered by Savane 3.1-cleanup