bugGNU Parallel - build and execute command lines from standard input in parallel - Bugs: bug #44357, Some large file support is broken

 
 

bug #44357: Some large file support is broken

Submitted by:  Ole Tange <tange>
Submitted on:  Mon 23 Feb 2015 10:16:06 PM UTC  
 
Category: NoneSeverity: 3 - Normal
Item Group: NoneStatus: Fixed
Privacy: PublicAssigned to: Ole Tange <tange>
Open/Closed: Closed

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

Tue 24 Feb 2015 01:29:37 AM UTC, comment #2:

Fixed in [c445232].

Ole Tange <tange>
Project AdministratorIn charge of this item.
Tue 24 Feb 2015 01:16:27 AM UTC, comment #1:

Good test:

(yes "`echo {1..100}`" | head -c 5000000000; echo FOO;yes "`echo {1..100}`" | head -c 3000000000;echo FOO;yes "`echo {1..100}`" | head -c 1000000000;) | ppar --pipe --recend FOO --block 1g -k LANG=c wc -c

Ole Tange <tange>
Project AdministratorIn charge of this item.
Mon 23 Feb 2015 10:16:06 PM UTC, original submission:

https://elefant.bifo.helmholtz-hzi.de/public.php?service=files&t=48fb2c2e7ba7ace340acf37ffe9803f3

zcat debug.maf.gz | parallel --halt-on-error --no-notice --gnu --pipe --recstart '# batch ' --recend '\n\n' 'cat > "$PARALLEL_SEQ".maf'

Results in 1.maf = 2GB - 4 KB.

(yes "`echo {1..100}`" | head -c 3G; echo FOO;yes "`echo {1..100}`" | head -c 3G;) | parallel --pipe --recend FOO LANG=c wc -c

gives: 2147479552, but should give 3 GB

Ole Tange <tange>
Project AdministratorIn charge of this item.

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by tange (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 2 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Tue 24 Feb 2015 01:29:37 AM UTCtangeStatusConfirmed=>Fixed
      Open/ClosedOpen=>Closed

    Back to the top


    Powered by Savane 3.1-cleanup1