bugmake - Bugs: bug #15182, Make uses too much memory

 
 

bug #15182: Make uses too much memory

Submitter:  None
Submitted:  Sat 10 Dec 2005 07:58:17 PM UTC
Votes: 1
 
Severity:  3 - Normal Item Group:  Bug
Status:  Fixed Privacy:  Public
Assigned to:  psmith Open/Closed:  Closed
Component Version:  3.80 Operating System:  POSIX-Based
Fixed Release:  3.82 Triage Status:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Sun 04 Oct 2009 10:09:37 PM UTC, comment #7: 

I believe that changes made in the next version of GNU make will alleviate this problem.  I'm having a lot of trouble getting GCC to build properly for me so I haven't been able to test it.

Paul D. Smith <psmith>
Group administrator
Wed 22 Feb 2006 07:02:25 PM UTC, comment #6: 

question: for this specific situation in gcj and GNU Classpath, are the dependency lines listed as they are in the makefile attached to this bug report? That is, like this:

targ1 targ2 targ3 ... : prereq1 prereq2 prereq3 ...

answer: Yes. The gnu-src-gcc.deps is literally from the gcc/libjava/classpath/lib/lists .dep files

Mark Wielaard <mark>
Tue 07 Feb 2006 02:35:43 PM UTC, comment #5: 

A complete fix for this problem will have to wait until after 3.81 is released.  I know how I want to solve it, but I'm trying hard to get 3.81 out the door and really need to clamp down on changes.  However, I'd like to fix this issue for gcj.  As you've noted, the code in 3.81 has changed quite a bit in this area (but not because of the hash_* functions; those were used in 3.80 as well--the change is because the code has been rearranged to support the secondary expansion feature).

So, let me ask a question: for this specific situation in gcj and GNU Classpath, are the dependency lines listed as they are in the makefile attached to this bug report?  That is, like this:

  targ1 targ2 targ3 ... : prereq1 prereq2 prereq3 ...

Or, are they broken out into multiple lines, like this:

  targ1: prereq1 prereq2 prereq3 ...
  targ2: prereq1 prereq2 prereq3 ...
  targ3: prereq1 prereq2 prereq3 ...
    ...

?  If the former, I can do a simple, quick fix for 3.81 that will solve the problem and then I can get a more complete solution for the next release.  If that latter... we'll need something much closer to the complete solution to fix the issue.

Paul D. Smith <psmith>
Group administrator
Thu 02 Feb 2006 11:15:45 PM UTC, comment #4: 

We are also using this patch for the make on builder.classpath.org which is the GNU Classpath autobuilder and regression tester. The patch is actually needed on that machine since it has "only" 512MB and would otherwise swap too much to be able to do  continuous builds.

Mark Wielaard <mark>
Thu 02 Feb 2006 05:25:18 PM UTC, comment #3: 

Hi,

Please consider inclusion of parts of this patch.

Make memory consumption is a real problem to build gcj-4.0 in Debian.  Please see http://bugs.debian.org/342726 for details, but this patch doesn't look too intrusive and helps solving the memory consumption issues by storing hashes of strings instead of strings in memory.

This patch was already sent at:
http://gcc.gnu.org/ml/gcc/2005-12/msg00231.html

and updated at:
http://gcc.gnu.org/ml/gcc/2006-02/msg00026.html

It was written against the stable version of make and only needs to be updated to use make's hash_* functions from CVS.

Cheers,

Loïc Minier <lool>
Thu 02 Feb 2006 02:45:28 PM UTC, comment #2: 

There are 2 typos in my first patch. I uploaded the fixed one.

Anonymous
Sat 10 Dec 2005 08:01:06 PM UTC, comment #1: 

GNU make 3.80 is a HUGE memory hog. It calls xstrdup to build
dependency list. gnu-src-gcc.deps in libjava has 3000+ targets depend
the same 3000+ files, whose filenames are more than 260K. For this
dependency alone, make takes 3000*260K == 761MB.

This patch is a quick hack. It reduces the memory from 1.6GB to around
600MB for "make -f gnu-src-gcc.deps". I got

[hjl@gnu-13 libjava]$ make
ulimit -v 1048576; \
/usr/bin/time ./make -f gnu-src-gcc.deps
make[1]: Entering directory `/export/home/hjl/bugs/make/libjava'
make[1]: * No rule to make target `../gnu/classpath/Configuration.java',
needed by `lists/gnu-src-gcc.stamp'.  Stop.
make[1]: Leaving directory `/export/home/hjl/bugs/make/libjava'
Command exited with non-zero status 2
67.10user 1.24system 1:16.63elapsed 89%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+140611minor)pagefaults 0swaps
make: * [all] Error 2

Anonymous
Sat 10 Dec 2005 07:58:17 PM UTC, original submission:  

In some cases, make uses too much memory. For example:

[hjl@gnu-d libjava]$ make
ulimit -v 1048576; \
/usr/bin/time make -f gnu-src-gcc.deps
make[1]: Entering directory `/export/home/hjl/bugs/make/libjava'
make[1]: * virtual memory exhausted.  Stop.
make[1]: Leaving directory `/export/home/hjl/bugs/make/libjava'
Command exited with non-zero status 2
5.94user 2.78system 0:22.33elapsed 39%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (279major+261891minor)pagefaults 0swaps
make: * [all] Error 2

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #2061:  make-3.80-memory-2.patch added by None (6KiB - text/x-patch - An updated patch)
file #2458:  make-3.80-memory-1.patch added by None (6KiB - text/x-patch - A quick hack)
file #2457:  bug.tar.bz2 added by None (30KiB - application/x-bzip - A testcase)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by psmith (Posted a comment)
  • -email is unavailable- added by psmith (Debian Bug Tracker)
  •  

    There is 1 vote so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 11 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2010-07-28 psmith Fixed Release4.0 3.82
    2009-10-04 psmith StatusNone Fixed
        Open/ClosedOpen Closed
        Fixed ReleaseNone 4.0
    2006-02-07 psmith Assigned toNone psmith
    2006-02-07 psmith Carbon-Copy- Added -email is unavailable-
    2006-02-02 lool Carbon-Copy- Added lool
    2006-02-02 None Attached File- Added make-3.80-memory-2.patch, #3356
    2005-12-13 mark Carbon-Copy- Added mark
    2005-12-10 None Attached File- Added make-3.80-memory-1.patch, #3179
    2005-12-10 None Attached File- Added bug.tar.bz2, #3178

    Back to the top

    Powered by Savane 3.13-f8d8.
    Corresponding source code