Thu 23 Jun 2011 03:02:31 PM UTC, comment #5:
I'm running killall in a separate terminal because parallel hangs:
$ seq 5 | parallel --load=2 echo
1
3
4
5
parallel: echo 2
$ seq 5 | parallel --load=2 --progress echo
Computers / CPU cores / Max jobs to run
1:local / 4 / 4
1
3
4
Computer:jobs running/jobs completed/%of started jobs/Average seconds to complete
5
local:1/4/100%/1.8s parallel: echo 2
$ seq 5 | parallel --load=2 -D echo
1 128
128 0
2 2048
2048 0
3 32768
32768 0
4 524288
524288 -1
Maxlen: 32768,524288,278528
5 278528
278528 -1
Maxlen: 32768,278528,155648
6 155648
155648 -1
Maxlen: 32768,155648,94208
7 94208
94208 0
Maxlen: 94208,155648,124928
8 124928
124928 0
Maxlen: 124928,155648,140288
9 140288
140288 -1
Maxlen: 124928,140288,132608
10 132608
132608 -1
Maxlen: 124928,132608,128768
11 128768
128768 0
Maxlen: 128768,132608,130688
12 130688
130688 0
Maxlen: 130688,132608,131648
13 131648
131648 -1
Maxlen: 130688,131648,131168
14 131168
131168 -1
Maxlen: 130688,131168,130928
15 130928
130928 0
Maxlen: 130928,131168,131048
16 131048
131048 0
Maxlen: 131048,131168,131108
17 131108
131108 -1
Maxlen: 131048,131108,131078
18 131078
131078 -1
Maxlen: 131048,131078,131063
19 131063
131063 0
Maxlen: 131063,131078,131070
20 131070
131070 0
Maxlen: 131070,131078,131074
21 131074
131074 -1
Maxlen: 131070,131074,131072
22 131072
131072 -1
Maxlen: 131070,131072,131071
23 131071
131071 0
Wanted procs: 4
MultifileQueue->empty
RecordQueue->empty
CommandLineQueue->empty
JobQueue->empty
MultifileQueue->empty
RecordQueue->empty
read 1
cmd_line->number_of_args 1
Time to fork 1 procs: 0 (processes so far: 1)
MultifileQueue->empty
RecordQueue->empty
CommandLineQueue->empty
JobQueue->empty
MultifileQueue->empty
RecordQueue->empty
read 2
cmd_line->number_of_args 1
Time to fork 2 procs: 0 (processes so far: 2)
MultifileQueue->empty
RecordQueue->empty
CommandLineQueue->empty
JobQueue->empty
MultifileQueue->empty
RecordQueue->empty
read 3
cmd_line->number_of_args 1
Time to fork 3 procs: 0 (processes so far: 3)
MultifileQueue->empty
RecordQueue->empty
CommandLineQueue->empty
JobQueue->empty
MultifileQueue->empty
RecordQueue->empty
read 4
cmd_line->number_of_args 1
Time to fork 4 procs: 0 (processes so far: 4)
RecordQueue-unget ''
Limited to procs: 4
Running jobs before on :: 0
No loadavg file: /root/.parallel/tmp/loadavg-11371-:Updating loadavg file/root/.parallel/tmp/loadavg-11371-:Reaper called 1
Reaper exit 1
Start draining
JobQueue->empty
Running jobs before on :: 0
New loadavg: 0.21Last update: 1308840997max_loadavg: : 2JobQueue->empty
Try starting a job on :
JobQueue->empty
Command to run on 'SSHLogin=HASH(0x17973180)': 'echo 1'
1 processes. Starting (1): echo 1
Started as seq 1
Job started on :
JobQueue->empty
Try starting a job on :
JobQueue->empty
Command to run on 'SSHLogin=HASH(0x17973180)': 'echo 2'
2 processes. Starting (2): echo 2
Reaper called 1
Reaper called 1
Reaper exit 1
died (0): 1>>joboutput echo 1
ERR:
OUT:
1
<<joboutput echo 1
Running jobs before on :: 0
New loadavg: 0.21Last update: 1308840997max_loadavg: : 2JobQueue->empty
Try starting a job on :
JobQueue->empty
Command to run on 'SSHLogin=HASH(0x17973180)': 'echo 3'
2 processes. Starting (3): echo 3
Started as seq 3
Job started on :
JobQueue->empty
Try starting a job on :
JobQueue->empty
Command to run on 'SSHLogin=HASH(0x17973180)': 'echo 4'
3 processes. Starting (4): echo 4
Started as seq 4
Job started on :
MultifileQueue->empty
RecordQueue->empty
CommandLineQueue->empty
JobQueue->empty
Try starting a job on :
MultifileQueue->empty
RecordQueue->empty
CommandLineQueue->empty
JobQueue->empty
MultifileQueue->empty
RecordQueue->empty
read 5
cmd_line->number_of_args 1
Command to run on 'SSHLogin=HASH(0x17973180)': 'echo 5'
4 processes. Starting (5): echo 5
Started as seq 5
Job started on :
MultifileQueue->empty 1
RecordQueue->empty 1
CommandLineQueue->empty 1
JobQueue->empty 1
Running jobs after on :: 3 of 4
died (0): 3>>joboutput echo 3
ERR:
OUT:
3
<<joboutput echo 3
Running jobs before on :: 2
New loadavg: 0.21Last update: 1308840997max_loadavg: : 2MultifileQueue->empty 1
RecordQueue->empty 1
CommandLineQueue->empty 1
JobQueue->empty 1
Running jobs after on :: 2 of 4
died (0): 4>>joboutput echo 4
ERR:
OUT:
4
<<joboutput echo 4
Running jobs before on :: 1
New loadavg: 0.21Last update: 1308840997max_loadavg: : 2MultifileQueue->empty 1
RecordQueue->empty 1
CommandLineQueue->empty 1
JobQueue->empty 1
Running jobs after on :: 1 of 4
Reaper exit 1
Reaper called 1
Reaper exit 1
Started as seq 2
Job started on :
MultifileQueue->empty 1
RecordQueue->empty 1
CommandLineQueue->empty 1
JobQueue->empty 1
Running jobs after on :: 2 of 4
Sleeping 0.22 millisecs
jobs running: 2 2 Memory usage:96768000
Sleeping 0.242 millisecs
Reaper called 1
Reaper exit 1
jobs running: 2 2 Memory usage:96768000
Sleeping 0.2662 millisecs
Reaper called 1
Reaper exit 1
jobs running: 2 2 Memory usage:96768000
Sleeping 0.29282 millisecs
Reaper called 1
Reaper exit 1
jobs running: 2 2 Memory usage:96768000
Sleeping 0.322102 millisecs
Reaper called 1
Reaper exit 1
jobs running: 2 2 Memory usage:96768000
Sleeping 0.3543122 millisecs
Reaper called 1
Reaper exit 1
jobs running: 2 2 Memory usage:96768000
Sleeping 0.38974342 millisecs
Reaper called 1
Reaper exit 1
jobs running: 2 2 Memory usage:96768000
Sleeping 0.428717762 millisecs
Reaper called 1
Reaper exit 1
jobs running: 2 2 Memory usage:96768000
Sleeping 0.4715895382 millisecs
Reaper called 1
died (0): 5>>joboutput echo 5
ERR:
OUT:
5
<<joboutput echo 5
Running jobs before on :: 1
New loadavg: 0.21Last update: 1308840997max_loadavg: : 2MultifileQueue->empty 1
RecordQueue->empty 1
CommandLineQueue->empty 1
JobQueue->empty 1
Running jobs after on :: 1 of 4
Reaper exit 1
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 0.51874849202 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 0.570623341222 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 0.6276856753442 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 0.69045424287862 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 0.759499667166483 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 0.835449633883131 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 0.918994597271444 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 1.01089405699859 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 1.11198346269845 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 1.22318180896829 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 1.34549998986512 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 1.48004998885163 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 1.6280549877368 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 1.79086048651048 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 1.96994653516153 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 2.16694118867768 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 2.38363530754545 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 2.62199883829999 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 2.88419872212999 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 3.17261859434299 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 3.48988045377729 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 3.83886849915502 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 4.22275534907052 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 4.64503088397757 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 5.10953397237533 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 5.62048736961286 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 6.18253610657415 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 6.80078971723157 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 7.48086868895472 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 8.2289555578502 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 9.05185111363522 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 9.95703622499874 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 10.9527398474986 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 12.0480138322485 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 13.2528152154733 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 14.5780967370207 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 16.0359064107227 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 17.639497051795 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 19.4034467569745 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 21.343791432672 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 23.4781705759392 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 25.8259876335331 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 28.4085863968864 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 31.249445036575 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 34.3743895402325 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 37.8118284942558 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 41.5930113436814 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 45.7523124780495 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 50.3275437258545 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 55.3602980984399 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 60.8963279082839 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 66.9859606991123 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 73.6845567690235 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 81.0530124459259 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 89.1583136905185 millisecs
parallel: echo 2
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 98.0741450595704 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 107.881559565527 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 118.66971552208 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 130.536687074288 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 143.590355781717 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 157.949391359889 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 173.744330495878 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 191.118763545465 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 210.230639900012 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 231.253703890013 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 254.379074279014 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 279.816981706916 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 307.798679877608 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 338.578547865368 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 372.436402651905 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 409.680042917096 millisecs
Reaper called 1
Reaper exit 1
jobs running: 1 1 Memory usage:96768000
Sleeping 450.648047208805 millisecs
|
Thu 23 Jun 2011 01:27:27 PM UTC, comment #2:
I can reproduce it reliably on three of four similar systems. Piping to stdin hangs almost all of the time; ::: hangs once in a while. I can give you (the last bits of) an strace before I CTRL-C it:
$ strace parallel --load=2 echo ::: 1 2 3 4 5
[...snip...]
select(0, NULL, NULL, NULL, {0, 798350}) = 0 (Timeout)
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigaction(SIGCHLD, {0x318e07de40, [], SA_RESTORER, 0x3185a0eb10}, {0x318e07de40, [], SA_RESTORER, 0x3185a0eb10}, 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigaction(SIGCHLD, {0x318e07de40, [], SA_RESTORER, 0x3185a0eb10}, {0x318e07de40, [], SA_RESTORER, 0x3185a0eb10}, 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
wait4(-1, 0x7fff725ceb84, WNOHANG, NULL) = -1 ECHILD (No child processes)
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigaction(SIGCHLD, {0x318e07de40, [], SA_RESTORER, 0x3185a0eb10}, {0x318e07de40, [], SA_RESTORER, 0x3185a0eb10}, 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigaction(SIGCHLD, {0x318e07de40, [], SA_RESTORER, 0x3185a0eb10}, {0x318e07de40, [], SA_RESTORER, 0x3185a0eb10}, 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
stat("/proc/2813/stat", {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
open("/proc/2813/stat", O_RDONLY) = 9
ioctl(9, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fff725ce740) = -1 ENOTTY (Inappropriate ioctl for device)
lseek(9, 0, SEEK_CUR) = 0
fstat(9, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
fcntl(9, F_SETFD, FD_CLOEXEC) = 0
read(9, "2813 (parallel) R 2812 2812 1209"..., 4096) = 226
close(9) = 0
select(0, NULL, NULL, NULL, {0, 878185}) = 0 (Timeout)
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigaction(SIGCHLD, {0x318e07de40, [], SA_RESTORER, 0x3185a0eb10}, {0x318e07de40, [], SA_RESTORER, 0x3185a0eb10}, 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigaction(SIGCHLD, {0x318e07de40, [], SA_RESTORER, 0x3185a0eb10}, {0x318e07de40, [], SA_RESTORER, 0x3185a0eb10}, 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
wait4(-1, 0x7fff725ceb84, WNOHANG, NULL) = -1 ECHILD (No child processes)
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigaction(SIGCHLD, {0x318e07de40, [], SA_RESTORER, 0x3185a0eb10}, {0x318e07de40, [], SA_RESTORER, 0x3185a0eb10}, 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigaction(SIGCHLD, {0x318e07de40, [], SA_RESTORER, 0x3185a0eb10}, {0x318e07de40, [], SA_RESTORER, 0x3185a0eb10}, 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
stat("/proc/2813/stat", {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
open("/proc/2813/stat", O_RDONLY) = 9
ioctl(9, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fff725ce740) = -1 ENOTTY (Inappropriate ioctl for device)
lseek(9, 0, SEEK_CUR) = 0
fstat(9, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
fcntl(9, F_SETFD, FD_CLOEXEC) = 0
read(9, "2813 (parallel) R 2812 2812 1209"..., 4096) = 226
close(9) = 0
select(0, NULL, NULL, NULL, {0, 966004}) = 0 (Timeout)
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigaction(SIGCHLD, {0x318e07de40, [], SA_RESTORER, 0x3185a0eb10}, {0x318e07de40, [], SA_RESTORER, 0x3185a0eb10}, 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigaction(SIGCHLD, {0x318e07de40, [], SA_RESTORER, 0x3185a0eb10}, {0x318e07de40, [], SA_RESTORER, 0x3185a0eb10}, 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
wait4(-1, 0x7fff725ceb84, WNOHANG, NULL) = -1 ECHILD (No child processes)
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigaction(SIGCHLD, {0x318e07de40, [], SA_RESTORER, 0x3185a0eb10}, {0x318e07de40, [], SA_RESTORER, 0x3185a0eb10}, 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigaction(SIGCHLD, {0x318e07de40, [], SA_RESTORER, 0x3185a0eb10}, {0x318e07de40, [], SA_RESTORER, 0x3185a0eb10}, 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
stat("/proc/2813/stat", {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
open("/proc/2813/stat", O_RDONLY) = 9
ioctl(9, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fff725ce740) = -1 ENOTTY (Inappropriate ioctl for device)
lseek(9, 0, SEEK_CUR) = 0
fstat(9, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
fcntl(9, F_SETFD, FD_CLOEXEC) = 0
read(9, "2813 (parallel) R 2812 2812 1209"..., 4096) = 226
close(9) = 0
select(0, NULL, NULL, NULL, {1, 62604}) = ? ERESTARTNOHAND (To be restarted)
parallel: SIGTERM received. No new jobs will be started.
parallel: Waiting for these 1 jobs to finish. Send SIGTERM again to stop now.
parallel: echo 1
|