bugmake - Bugs: bug #23960, Crash when MAKEFILE_LIST becomes...

 
 

bug #23960: Crash when MAKEFILE_LIST becomes too long.

Submitter:  Timothy N Murphy <tnmurphy>
Submitted:  Tue 29 Jul 2008 11:52:53 AM UTC
   
 
Severity:  3 - Normal Item Group:  Bug
Status:  Fixed Privacy:  Public
Assigned to:  psmith Open/Closed:  Closed
Component Version:  3.81 Operating System:  MS Windows
Fixed Release:  3.82 Triage Status:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Sat 01 Aug 2009 11:55:14 PM UTC, comment #1: 

We definitely don't want to be using alloca() to hold variable values since they can get very large; alloca() is only for things which we know will be small.

Fixed to use xmalloc'd memory.

Paul D. Smith <psmith>
Group administrator
Tue 29 Jul 2008 11:52:53 AM UTC, original submission:  

I am working on a project which generates huge makefiles. These makefiles include others many many times - there are perhaps 12,000 instances of the "include" statement at a rough guess.

We found that GNU Make crashed on Windows and it took us a while to realise that MAKEFILE_LIST was the problem.  We worked around it by setting MAKEFILE_LIST to "blank" after every include statement, since we don't need its value.

It turns out that the routine in make that appends to variables uses alloca and it does appends in such a way that I imagine the mechanism for stack expansion (the guard page) is being defeated.

The location is in do_variable_definition():

            oldlen = strlen (v->value);
            vallen = strlen (val);
            p = (char *) alloca (oldlen + 1 + vallen + 1);
            bcopy (v->value, p, oldlen);
            p[oldlen] = ' ';
            bcopy (val, &p[oldlen + 1], vallen + 1);

This copies the old value to the alloca'd block then adds a space and copies the new value.

The problem is that stack grows down so if 'old_len' is larger than the guard page, then the first memory access by bcopy to 'p' could occur below the guard page.  This will not trigger the expansion of the stack.

The problem might happen with any variable that grows sufficiently large.

It's not easy to come up with a perfect fix for this and the obvious ones such as using the heap would slow down the common case.  There are also no memory copy routines that work from high to low addresses which would obviously be ideal.

An initial suggestion might be something like this:
            bcopy (val, &p[oldlen + 1], vallen + 1);
            p[oldlen] = ' ';
            bcopy (v->value, p, oldlen);

Where by swapping the bcopy()s one improves the chance that the guard page will be triggered (initialise high to low).  One might even split the bcopies into loops that always copy less than the minimum page size and move down.

Our use case is extreme and there is a workaround so I don't suggest that there is an immediate need to change this but it is worth noting in case anyone else has the same problem.

Timothy N Murphy <tnmurphy>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #18500:  variable_assign_crash.mk added by tnmurphy (6KiB - application/octet-stream - A test case to demonstrate. Crashes with mingw-make on win32)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by psmith (Posted a comment)
  • -email is unavailable- added by tnmurphy (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 6 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2010-07-28 psmith Fixed Release4.0 3.82
    2009-08-01 psmith StatusNone Fixed
        Assigned toNone psmith
        Open/ClosedOpen Closed
        Fixed ReleaseNone 4.0
    2009-07-30 tnmurphy Attached File- Added variable_assign_crash.mk, #18500

    Back to the top

    Powered by Savane 3.13-f810.
    Corresponding source code