bugmake - Bugs: bug #15182, Make uses too much memory

 
 

bug #15182: Make uses too much memory

Submitted by:  None
Submitted on:  Sat 10 Dec 2005 07:58:17 PM UTC  
Votes:  1  
 
Severity: 3 - NormalItem Group: Bug
Status: FixedPrivacy: Public
Assigned to: Paul D. Smith <psmith>Open/Closed: Closed
Component Version: 3.80Operating System: POSIX-Based
Fixed Release: 3.82Triage Status: None

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

(Jump to the original submission Jump to the original submission)

Sun 04 Oct 2009 10:09:37 PM UTC, comment #7:

I believe that changes made in the next version of GNU make will alleviate this problem. I'm having a lot of trouble getting GCC to build properly for me so I haven't been able to test it.

Paul D. Smith <psmith>
Project AdministratorIn charge of this item.
Wed 22 Feb 2006 07:02:25 PM UTC, comment #6:

question: for this specific situation in gcj and GNU Classpath, are the dependency lines listed as they are in the makefile attached to this bug report? That is, like this:

targ1 targ2 targ3 ... : prereq1 prereq2 prereq3 ...

answer: Yes. The gnu-src-gcc.deps is literally from the gcc/libjava/classpath/lib/lists .dep files

Mark Wielaard <mark>
Tue 07 Feb 2006 02:35:43 PM UTC, comment #5:

A complete fix for this problem will have to wait until after 3.81 is released. I know how I want to solve it, but I'm trying hard to get 3.81 out the door and really need to clamp down on changes. However, I'd like to fix this issue for gcj. As you've noted, the code in 3.81 has changed quite a bit in this area (but not because of the hash_* functions; those were used in 3.80 as well--the change is because the code has been rearranged to support the secondary expansion feature).

So, let me ask a question: for this specific situation in gcj and GNU Classpath, are the dependency lines listed as they are in the makefile attached to this bug report? That is, like this:

targ1 targ2 targ3 ... : prereq1 prereq2 prereq3 ...

Or, are they broken out into multiple lines, like this:

targ1: prereq1 prereq2 prereq3 ...
targ2: prereq1 prereq2 prereq3 ...
targ3: prereq1 prereq2 prereq3 ...
...

? If the former, I can do a simple, quick fix for 3.81 that will solve the problem and then I can get a more complete solution for the next release. If that latter... we'll need something much closer to the complete solution to fix the issue.

Paul D. Smith <psmith>
Project AdministratorIn charge of this item.
Thu 02 Feb 2006 11:15:45 PM UTC, comment #4:

We are also using this patch for the make on builder.classpath.org which is the GNU Classpath autobuilder and regression tester. The patch is actually needed on that machine since it has "only" 512MB and would otherwise swap too much to be able to do continuous builds.

Mark Wielaard <mark>
Thu 02 Feb 2006 05:25:18 PM UTC, comment #3:

Hi,

Please consider inclusion of parts of this patch.

Make memory consumption is a real problem to build gcj-4.0 in Debian. Please see http://bugs.debian.org/342726 for details, but this patch doesn't look too intrusive and helps solving the memory consumption issues by storing hashes of strings instead of strings in memory.

This patch was already sent at:
http://gcc.gnu.org/ml/gcc/2005-12/msg00231.html

and updated at:
http://gcc.gnu.org/ml/gcc/2006-02/msg00026.html

It was written against the stable version of make and only needs to be updated to use make's hash_* functions from CVS.

Cheers,

Loïc Minier <lool>
Thu 02 Feb 2006 02:45:28 PM UTC, comment #2:

There are 2 typos in my first patch. I uploaded the fixed one.

Anonymous
Sat 10 Dec 2005 08:01:06 PM UTC, comment #1:

GNU make 3.80 is a HUGE memory hog. It calls xstrdup to build
dependency list. gnu-src-gcc.deps in libjava has 3000+ targets depend
the same 3000+ files, whose filenames are more than 260K. For this
dependency alone, make takes 3000*260K == 761MB.

This patch is a quick hack. It reduces the memory from 1.6GB to around
600MB for "make -f gnu-src-gcc.deps". I got

[hjl@gnu-13 libjava]$ make
ulimit -v 1048576; \
/usr/bin/time ./make -f gnu-src-gcc.deps
make[1]: Entering directory `/export/home/hjl/bugs/make/libjava'
make[1]: *** No rule to make target `../gnu/classpath/Configuration.java',
needed by `lists/gnu-src-gcc.stamp'. Stop.
make[1]: Leaving directory `/export/home/hjl/bugs/make/libjava'
Command exited with non-zero status 2
67.10user 1.24system 1:16.63elapsed 89%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+140611minor)pagefaults 0swaps
make: *** [all] Error 2

Anonymous
Sat 10 Dec 2005 07:58:17 PM UTC, original submission:

In some cases, make uses too much memory. For example:

[hjl@gnu-d libjava]$ make
ulimit -v 1048576; \
/usr/bin/time make -f gnu-src-gcc.deps
make[1]: Entering directory `/export/home/hjl/bugs/make/libjava'
make[1]: *** virtual memory exhausted. Stop.
make[1]: Leaving directory `/export/home/hjl/bugs/make/libjava'
Command exited with non-zero status 2
5.94user 2.78system 0:22.33elapsed 39%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (279major+261891minor)pagefaults 0swaps
make: *** [all] Error 2

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

Attached Files
file #2061:  make-3.80-memory-2.patch added by None (6KiB - text/x-patch - An updated patch)
file #2458:  make-3.80-memory-1.patch added by None (6KiB - text/x-patch - A quick hack)
file #2457:  bug.tar.bz2 added by None (30KiB - application/x-bzip - A testcase)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by psmith (Posted a comment)
  • -unavailable- added by psmith (Debian Bug Tracker)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 1 encouragement so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 11 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Wed 28 Jul 2010 06:21:38 AM UTCpsmithFixed Release4.0=>3.82
    Sun 04 Oct 2009 10:09:37 PM UTCpsmithStatusNone=>Fixed
      Open/ClosedOpen=>Closed
      Fixed ReleaseNone=>4.0
    Tue 07 Feb 2006 02:35:43 PM UTCpsmithAssigned toNone=>psmith
    Tue 07 Feb 2006 02:31:04 PM UTCpsmithCarbon-Copy-=>Added -unavailable-
    Thu 02 Feb 2006 05:25:18 PM UTCloolCarbon-Copy-=>Added lool
    Thu 02 Feb 2006 02:43:58 PM UTCNoneAttached File-=>Added make-3.80-memory-2.patch, #3356
    Tue 13 Dec 2005 02:48:39 PM UTCmarkCarbon-Copy-=>Added mark
    Sat 10 Dec 2005 08:01:06 PM UTCNoneAttached File-=>Added make-3.80-memory-1.patch, #3179
    Sat 10 Dec 2005 07:58:17 PM UTCNoneAttached File-=>Added bug.tar.bz2, #3178

    Back to the top


    Powered by Savane 3.1-cleanup1