bugGNU Parallel - Bugs: bug #63486, parallel: Error: Argument list too...

 
 

bug #63486: parallel: Error: Argument list too long error

Submitter:  None
Submitted:  Thu 08 Dec 2022 09:05:03 AM UTC
   
 
Category:  None Severity:  3 - Normal
Item Group:  None Status:  Postponed
Privacy:  Public Assigned to:  tange
Open/Closed:  Open
* Mandatory Fields

Add a New Comment Rich Markup
   

Tue 07 Mar 2023 05:01:44 PM UTC, comment #3: 

It is really this bug that blocks it:

https://savannah.gnu.org/bugs/?44961


Ole Tange <tange>
Group administrator
Fri 09 Dec 2022 03:12:43 PM UTC, comment #2: 

Thanks for your reply.

I do use --env/--session when possible but still it becomes too much of a burden when there are many environment items. env_parallel usually "just works" but this too long error is the one thing I have pain dealing with. I'm very glad to hear there is possibility of fixing this limitation. I hope that it gets implemented!

 Anonymous
Fri 09 Dec 2022 11:57:35 AM UTC, comment #1: 

The problem is covered in:
https://www.gnu.org/software/parallel/env_parallel.html#environment-space

The workaround is to use --env or --session.

When you get it working, please file a report on how 'man env_parallel' could be updated so it would cover your solution.

The long term solution would be to make the receiving process a perl process that reads from STDIN. This means the environment is not limited by execve. It makes it harder to implement --pipe, but not impossible. So while I have a working design in my head, getting it actually implemented is harder.

But it is good to know that there are people who are hit by the limit and who would benefit from this.

 Anonymous
Thu 08 Dec 2022 09:05:03 AM UTC, original submission:  

I've been using env_parallel for a while now and it is great piece of software but the biggest issue with it is the Command line too long/Argument list too long error. I've read everything online and the only way I saw that could maybe fix it requires some hacky modification of the Linux kernel.

Is there really nothing else that can be done about it? Is there a way you can use some other internal function instead of IPC::open3 and prevent invoking execve which seems to be the issue? I've tried some workarounds in terms of feeding input or using --env to get around this but it isn't always possible to work around.

Thanks for all your great work.

 Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by tange (Posted a comment)

    •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 2 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2023-03-07 tange StatusNone Postponed
        Assigned toNone tange

    Back to the top

    Copyright © 2024 Free Software Foundation, Inc.
    Verbatim copying and distribution of this entire article is permitted in any medium, provided this notice is preserved.
    The Levitating, Meditating, Flute-playing Gnu logo is a GNU GPL'ed image provided by the Nevrax Design Team.
    Page source code

    Powered by Savane 3.13-54b4.
    Corresponding source code