bugGNU Parallel - Bugs: bug #63486, parallel: Error: Argument list too...

 
 

bug #63486: parallel: Error: Argument list too long error

Submitter:  None
Submitted:  Thu 08 Dec 2022 09:05:03 AM UTC
   
 
Category:  None Severity:  3 - Normal
Priority:  5 - Normal Item Group:  None
Status:  Postponed Privacy:  Public
Assigned to:  tange Open/Closed:  Open
* Mandatory Fields

Post a Comment

Add a New Comment Rich Markup
   

Discussion

Tue 07 Mar 2023 05:01:44 PM UTC, comment #3: 

It is really this bug that blocks it:

https://savannah. ... u.org/bugs/?44961


Ole Tange <tange>
Group administrator
Fri 09 Dec 2022 03:12:43 PM UTC, comment #2: 

Thanks for your reply.

I do use --env/--session when possible but still it becomes too much of a burden when there are many environment items. env_parallel usually "just works" but this too long error is the one thing I have pain dealing with. I'm very glad to hear there is possibility of fixing this limitation. I hope that it gets implemented!

Anonymous
Fri 09 Dec 2022 11:57:35 AM UTC, comment #1: 

The problem is covered in:
https://www.gnu.o ... environment-space

The workaround is to use --env or --session.

When you get it working, please file a report on how 'man env_parallel' could be updated so it would cover your solution.

The long term solution would be to make the receiving process a perl process that reads from STDIN. This means the environment is not limited by execve. It makes it harder to implement --pipe, but not impossible. So while I have a working design in my head, getting it actually implemented is harder.

But it is good to know that there are people who are hit by the limit and who would benefit from this.

Anonymous
Thu 08 Dec 2022 09:05:03 AM UTC, original submission:  

I've been using env_parallel for a while now and it is great piece of software but the biggest issue with it is the Command line too long/Argument list too long error. I've read everything online and the only way I saw that could maybe fix it requires some hacky modification of the Linux kernel.

Is there really nothing else that can be done about it? Is there a way you can use some other internal function instead of IPC::open3 and prevent invoking execve which seems to be the issue? I've tried some workarounds in terms of feeding input or using --env to get around this but it isn't always possible to work around.

Thanks for all your great work.

Anonymous

 

Attached Files

This item currently has no attached files.

(Note: upload size limit is set to 4.0MiB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

 

Dependencies

This item does not depend on any other items.

No items depend on this one.

 

Mail Notification Carbon-Copy List

Carbon-Copy List
  • -email is unavailable- added by tange (Posted a comment)
  •  

    Votes

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    History

    Follow 2 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2023-03-07 tange StatusNone Postponed
        Assigned toNone tange

    Back to the top

    Powered by Savane 3.16-11ef.
    Corresponding source code