bugGNU Parallel - build and execute command lines from standard input in parallel - Bugs: bug #36456, spawn sshs slower to avoid htting...


bug #36456: spawn sshs slower to avoid htting MaxStartup

Submitted by:  Ole Tange <tange>
Submitted on:  Sat May 12 23:38:00 2012  
Category: NoneSeverity: 3 - Normal
Item Group: NoneStatus: Fixed
Privacy: PublicAssigned to: Ole Tange <tange>
Open/Closed: Closed

Add a New Comment (Rich MarkupRich Markup):

You are not logged in

Please log in, so followups can be emailed to you.


Tue Jul 2 19:48:24 2013, comment #4:

Probably fixed in 20130630 alpha.

Ole Tange <tange>
Project AdministratorIn charge of this item.
Mon Aug 6 23:04:04 2012, comment #3:

Using --joblog we can run ssh to all hosts in parallel twice and get the timings with ms accuracity.

It requires --plain to work as we need to be able to ignore all --profile/$PARALLEL etc.

Ole Tange <tange>
Project AdministratorIn charge of this item.
Mon Aug 6 23:03:56 2012, comment #2:

Auto master connection does not solve this.


Ole Tange <tange>
Project AdministratorIn charge of this item.
Tue May 22 23:30:29 2012, comment #1:

We also need to see if this can be fixed using auto master connection

Ole Tange <tange>
Project AdministratorIn charge of this item.
Sat May 12 23:38:00 2012, original submission:

sshd has MaxStartup which limits how many ssh connections that can be in login state (typically 10).

To avoid hitting that limit GNU Parallel should wait a little between spawning another ssh to the same host. The time to wait could be the time it takes to run:

ssh remote echo foo

as that will be a full login. It should be run twice and the first be ignored, as DNS may account for the most of the delay.

Maybe a better wait time is the above time / 10 (for MaxStartup) * 2 (for 100% error margin).

To time this use:

use Time::HiRes qw ( time );

time() without hires will give time in seconds and if added 1 second it will give a pessimistic time.


time ssh m echo foo
sleep 0.004;ssh m echo & # This works

time ssh l echo foo
sleep 0.1;ssh l echo & # This works

time ssh n echo
sleep 0.02;ssh n echo & # This works

time ssh n2 echo
sleep 0.03;ssh n2 echo & # This works

Ole Tange <tange>
Project AdministratorIn charge of this item.


(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):

No files currently attached


Depends on the following items: None found

Items that depend on this one: None found


Carbon-Copy List
  • -unavailable- added by tange (Submitted the item)

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.


    Please enter the title of George Orwell's famous dystopian book (it's a date):



    Follow 3 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Tue Jul 2 19:48:24 2013tangeStatusNone=>Fixed
    Mon Aug 6 23:03:56 2012tangeAssigned toNone=>tange

    Back to the top

    Powered by Savane 3.1-cleanup1