bugGNU Parallel - Bugs: bug #39554, Feature request: line buffered...

 
 

bug #39554: Feature request: line buffered output

Submitter:  Ole Tange <tange>
Submitted:  Tue 23 Jul 2013 09:50:59 AM UTC
   
 
Category:  None Severity:  1 - Wish
Item Group:  None Status:  Fixed
Privacy:  Public Assigned to:  tange
Open/Closed:  Closed
* Mandatory Fields

Add a New Comment Rich Markup
   

Tue 30 Jul 2013 09:17:03 PM UTC, comment #2: 

Fixed in [5b16a48] and [df204a6].

Ole Tange <tange>
Group administrator
Thu 25 Jul 2013 02:05:18 PM UTC, comment #1: 

If non blocking read does not work on old systems try:

$pos_now = tell $in_fd;
# Seek to the end
seek $in_fd, 0, 1;
$pos_end = tell $in_fd;
seek $in_fd, $pos_now, 0;
if($pos_now != $pos_end) {
   read $pos_end - $pos_now;
   $pos_now = $pos_end;
}

Ole Tange <tange>
Group administrator
Tue 23 Jul 2013 09:50:59 AM UTC, original submission:  

For some jobs it will be nice to get output from the jobs while the jobs are running.

-u will do that but may mix up the lines.

So what is wanted is a non-mixing way to get full lines from each system.

The idea being that instead of calling print when a child dies, the file handles are read non-blocking periodically:

for $fd {
  $buf .= read_non_block($fd,1_000_000);
  $last_newline = index $buf,"\n",BACKWARDS;
  if $last_newline:
     print substr($buf,$last_newline);
     substr($buf,$last_newline)="";
  }
}

When the job dies: Read the rest and print it.



Ole Tange <tange>
Group administrator

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by tange (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 2 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2013-07-30 tange StatusConfirmed Fixed
        Open/ClosedOpen Closed

    Back to the top

    Powered by Savane 3.13-758e.
    Corresponding source code