bugGNU Parallel - build and execute command lines from standard input in parallel - Bugs: bug #50675, Crash with resume following...

 
 

bug #50675: Crash with resume following retry-failed and --eta

Submitted by:  pbugrep <pbugrep>
Submitted on:  Wed 29 Mar 2017 11:50:06 PM UTC  
 
Category: NoneSeverity: 3 - Normal
Item Group: NoneStatus: Need Info
Privacy: PublicAssigned to: Ole Tange <tange>
Open/Closed: Closed

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

Wed 03 May 2017 07:24:52 AM UTC, comment #3:

Given the information I am not able to reproduce the bug.

When you have a MCVE then feel free to re-open this bug.

Ole Tange <tange>
Project AdministratorIn charge of this item.
Sat 01 Apr 2017 03:06:01 PM UTC, comment #2:

Your repro steps should work, but I'd guess the issue was a corrupted statefile. Maybe due to unclean kill signal handling. Unfortunately, it looks like I didn't preserve that statefile.

It's possible that using a large number of ssh hosts (like 150) and large # of job lines (like 200k) makes it more likely to trigger.

pbugrep <pbugrep>
Thu 30 Mar 2017 11:05:44 AM UTC, comment #1:

Thanks for your error report. Unfortunately I cannot reproduce the problem.

Maybe I do not understand your error report correctly?

I tried reproducing your error with this, but failed.

IDFILE=`tempfile`
IPSFILE=`tempfile`

CMD='echo $$ >>/tmp/pids; sleep 3'
seq 8 > $IDFILE
echo localhost > $IPSFILE
>/tmp/pids

parallel --eta -a $IDFILE --env PATH --sshloginfile $IPSFILE --jobs 9 --joblog y.state "$CMD" &
sleep 3
kill -9 `head -n1 /tmp/pids`
wait
parallel --eta -a $IDFILE --env PATH --sshloginfile $IPSFILE --jobs 9 --retry-failed --joblog y.state "$CMD"
parallel --eta -a $IDFILE --env PATH --sshloginfile $IPSFILE --jobs 9 --resume --joblog y.state "$CMD"

Please post full code that triggers the error. Maybe you can simply adapt the above?

Regarding the sporadic bug: File it as a new bug, so we can track it there - I will probably be needing more info.

Ole Tange <tange>
Project AdministratorIn charge of this item.
Wed 29 Mar 2017 11:50:06 PM UTC, original submission:

I believe this came about from:

1.running a parallel job, `parallel --eta -a $IDFILE --env PATH --sshloginfile $IPSFILE --jobs 9 --joblog y.state cmd`
2. aborting it via kill <oldest pid> (this isn't documented anywhere, but presumably tells it to shut down cleanly?)
3. running --retry-failed
4. running `parallel --eta -a $IDFILE --env PATH --sshloginfile $IPSFILE --jobs 9 --resume --joblog y.state cmd`

results in:

Illegal division by zero at /usr/local/bin/parallel line 2759, <GEN0> line 200000.

2759: sprintf("%s:%d/%d/%d%%/%.1fs ",
$workerno{$_}, $running, $completed,
($running+$completed)*100
/ $Global::total_started, $time);

GNU parallel 20170322

There's also a sporadic bug with parallel failing to use all ssh hosts. In a jobfile of 200k entries with 150 hosts, only 50 (always the same ones) were being utilized when I did --jobs 300%.

pbugrep <pbugrep>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by tange (Posted a comment)
  • -unavailable- added by pbugrep (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 3 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Wed 03 May 2017 07:24:52 AM UTCtangeOpen/ClosedOpen=>Closed
    Thu 30 Mar 2017 11:05:44 AM UTCtangeStatusNone=>Need Info
      Assigned toNone=>tange

    Back to the top


    Powered by Savane 3.1-cleanup1