Thu 20 Jan 2011 06:45:55 PM UTC, original submission:
The docs say that --eta "forces GNU parallel to read all jobs before starting to find the number of jobs." It does not appear to do this, or if it does, the computed ETA is not accurate. In either case, the total number of jobs is never displayed, which would be useful.
In this example, on RHEL 5.5 x86_64 (perl 5.8.8), the ETA always says 0s, when parallel should easily be able to determine that jobs take 5 seconds:
-----------------------
$ time seq 0 5 | parallel -P2 --eta "echo {} > /dev/null; sleep 5"
Use of uninitialized value in scalar assignment at /usr/local/bin/parallel line 1635, <STDIN> line 1.
Use of uninitialized value in scalar assignment at /usr/local/bin/parallel line 1635, <STDIN> line 2.
Computers / CPU cores / Max jobs to run
1:local / 4 / 2
Computer:jobs running/jobs completed/%of started jobs/Average seconds to complete
ETA: 0s local:0/6/100%/2.7s
real 0m16.305s
user 0m0.135s
sys 0m2.010s
------------------------
Second, the average seconds to complete is not correct, which may be the root problem in computing the ETA. It currently appears to be wallclock time / total jobs, whereas it should be the sum of the wallclock times for each job / total jobs. I.e., each job should have a separate wallclock timer, and these should be summed. E.g., if 3 jobs take, in parallel, 1, 2 and 3 seconds, the average seconds should be (1 + 2 + 3) / 3 = 2, and not 3 / 2 = 1.5. "Seconds for a (serial) job" is what users are familiar with seeing and using as a rule of thumb and is more intuitive to reason with.
Putting these together, if GNU parallel knows the total number of jobs and the average time, the most useful progress display would be
completed jobs / total jobs / % complete / Avg. time
consistent with the familiar division-style usage "I've done 3/5 jobs."
|