bugGNU Octave - Bugs: bug #53943, memory usage explodes on a...

 
 

bug #53943: memory usage explodes on a javaMethod call

Submitter:  A.R. Burgers <arb>
Submitted:  Sat 19 May 2018 02:07:30 PM UTC
   
 
Category:  Interpreter Severity:  2 - Minor
Priority:  5 - Normal Item Group:  Performance
Status:  Works For Me Assigned to:  None
Originator Name:  Open/Closed:  * Closed
Release:  * dev Operating System:  * GNU/Linux
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Mon 20 Aug 2018 02:04:16 PM UTC, comment #4: 

for the record:

  • there is no Octave bug here.


  • The java VM heap size can be controlled in octave with a java.opts file as documented in the manual section "Passing parameters to the JVM"


A.R. Burgers <arb>
Tue 05 Jun 2018 01:18:21 AM UTC, comment #3: 

Do you agree that Java is behaving as documented and there is no Octave bug here?

Mike Miller <mtmiller>
Group Member
Mon 21 May 2018 07:18:06 PM UTC, comment #2: 

Here is I think a relevant discussion of this topic:

https://stackoverflow.com/questions/4667483/how-is-the-default-java-heap-size-determined

The 35Gb is 1/4 of the physical memory of 145Gb, which is the default MaxHeapSize


java -XX:+PrintFlagsFinal -version | grep HeapSize
    uintx ErgoHeapSizeLimit                         = 0               {product}
    uintx HeapSizePerGCThread                       = 87241520        {product}
    uintx InitialHeapSize                          := 2147483648      {product}
    uintx LargePageHeapSizeThreshold                = 134217728       {product}
    uintx MaxHeapSize                              := 32126271488     {product}
java version "1.7.0_161"
Java(TM) SE Runtime Environment (build 1.7.0_161-b13)
Java HotSpot(TM) 64-Bit Server VM (build 24.161-b13, mixed mode)


A.R. Burgers <arb>
Sun 20 May 2018 11:45:45 PM UTC, comment #1: 

This looks to me like the expected behavior of Java, the amount of memory that Java uses is outside of Octave's control.

FWIW I see the following numbers on my system.

  • octave-cli --norc uses about 650 MiB
  • octave --norc uses about 1,322 MiB
  • a standalone C program that creates a Java VM uses about 6,584 MiB
  • octave-cli --norc after loading a Java VM uses about 8,387 MiB
  • octave --norc after loading a Java VM uses about 9.037 MiB


The baseline memory difference between octave-cli and octave-gui is about 650 MiB. This is the same before and after Java is loaded.

The memory difference on my system due to starting a Java VM in Octave is about 7,730 MiB. This is slightly larger than the memory used by a simple C program that just creates a Java VM, but not much different.

So on my system I see a large increase of about 7.7 GiB when Octave creates a Java VM, but not the 34 GiB increase that you are seeing. This may be due to system configuration differences or a different version or build of Java.

You might try measuring the memory usage of a simple Java hello world program. On my system with the same version of Java, I see a memory usage of about 6.7 GiB, which is consistent with the other examples I tested.

Mike Miller <mtmiller>
Group Member
Sat 19 May 2018 02:07:30 PM UTC, original submission:  

with this script, that basically has one line invoking a javaMethod,


ver = version
hg_id = __octave_config_info__.hg_id

m_before = memory
dynamic_path = javaMethod ("getClassPath", "org.octave.ClassHelper")
m_after = memory


The memory.m file can be found here:

https://github.com/octave-de/macgyver_utils/blob/master/memory.m

The script outputs:


ver = 5.0.0
hg_id = 75fef3484d12+
m_before =

  scalar structure containing the fields:

    total =  660.15
    rss =  63.684
    shared =  28.328
    data =  194.57
    physical =  145319.24609

dynamic_path =
m_after =

  scalar structure containing the fields:

    total =  34653.17969
    rss =  91.375
    shared =  35.656
    data =  34165.11328
    physical =  145319.24609


Memory foot print jumps to 35 Gb. The jump is not limited to dev.

I noticed this phenomenon during an xlsopen from the io package, and narrowed it down to this line.

A.R. Burgers <arb>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by mtmiller (Posted a comment)
  • -email is unavailable- added by arb (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 3 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2018-06-06 mtmiller Open/ClosedOpen Closed
    2018-05-20 mtmiller Severity3 - Normal 2 - Minor
        StatusNone Works For Me

    Back to the top

    Powered by Savane 3.13-f8d8.
    Corresponding source code