bugmake - Bugs: bug #45763, Split args along MAX_ARG_STRLEN...

 
 

bug #45763: Split args along MAX_ARG_STRLEN for linux/posix

Submitted by:  None
Submitted on:  Fri Aug 14 19:06:31 2015  
Votes:  1  
 
Severity: 3 - NormalItem Group: Bug
Status: NonePrivacy: Public
Assigned to: NoneOpen/Closed: Open
Component Version: 4.1Operating System: POSIX-Based
Fixed Release: NoneTriage Status: None

Add a New Comment (Rich MarkupRich Markup):
   

You are not logged in

Please log in, so followups can be emailed to you.

 

Wed Feb 8 22:02:33 2017, comment #1:

I've rebased the patch to the latest git master as of this writing.

Keshav Kini <kini>
Fri Aug 14 19:06:31 2015, original submission:

When you run a make command where one of the targets has a sufficiently large enough arg list, you get:

make: execvp: /bin/sh: Argument list too long

This is because in GNU make they don't split arguments to /bin/sh in their $(shell) directive, so it's just /bin/sh -c "everything". This is running into MAX_ARG_STRLEN, and the debian patch definitely fixes it.

On debian they patch this to split along the pagesize so you can execute things of arbitrary length arguments. I've attached the patch that debian uses in their current package of make (4.0). My limits.h has things like stacksize set to unlimited, which means the following:

; getconf ARG_MAX
4611686018427387903
; cat /usr/include/linux/limits.h | grep ARG_MAX
#define ARG_MAX 131072 /* # bytes of args + environ for exec() */
; grep -HIins MAX_ARG_STRLEN /usr/include/linux/binfmts.h
/usr/include/linux/binfmts.h:10: * execve() system call. MAX_ARG_STRLEN is essentially random but serves to
/usr/include/linux/binfmts.h:14:#define MAX_ARG_STRLEN (PAGE_SIZE * 32)
; cat > test.c
#include <stdio.h>
#include <unistd.h>
int
main() {
printf("ARG_MAX: %ld\n", sysconf(_SC_ARG_MAX));
printf("PAGESIZE: %ld\n", sysconf(_SC_PAGESIZE));
}
^D
; gcc test.c && ./a.out
ARG_MAX: 4611686018427387903
PAGESIZE: 4096
;

Steps to reproduce:
Create a makefile with the following contents:

---
ALL_FILES := $(shell / -type f)
ECHO_FILES := echo $(ALL_FILES) | fmt -1

all: $(shell /bin/bash $(ECHO_FILES))
ls /
---

Name it 'makefile.test' or something, and then run:

make -f makefile.test all

And you'll see it print the error message.

Fix:

Debian has a patch that fixes this behavior, see my attachment.

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

Attached Files
file #39706:  0001-Patch-to-fix-large-command-line.patch added by kini (4kB - text/x-patch - Rebased patch to latest git master (2017-02-08))
file #34646:  0009-handle_excessive_command_length-Patch-to-fix-large-c.patch added by None (4kB - text/x-patch - debian patch to make for long argument)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by kini (Voted in favor of this item)
  • -unavailable- added by kini (Updated the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 1 encouragement so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 3 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Thu Feb 9 18:58:07 2017kiniCarbon-Copy-=>Added kini
    Wed Feb 8 22:01:29 2017kiniAttached File-=>Added 0001-Patch-to-fix-large-command-line.patch, #39706
    Fri Aug 14 19:06:31 2015NoneAttached File-=>Added 0009-handle_excessive_command_length-Patch-to-fix-large-c.patch, #34646

    Back to the top


    Powered by Savane 3.1-cleanup1