bugGNU Octave - Bugs: bug #42044, octave 3.8.1: huge memory use on...

 
 

bug #42044: octave 3.8.1: huge memory use on pkg load io

Submitted by:  None
Submitted on:  Fri 04 Apr 2014 07:18:40 AM UTC  
 
Category: Octave Forge PackageSeverity: 3 - Normal
Priority: 5 - NormalItem Group: Performance
Status: FixedAssigned to: None
Originator Name: A.R. BurgersOriginator Email: -unavailable-
Open/Closed: ClosedRelease: 3.8.1
Operating System: GNU/Linux

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

(Jump to the original submission Jump to the original submission)

Thu 17 Apr 2014 07:25:17 PM UTC, comment #9:

I've uploaded a new io pkg version (2.2.0) to the release tracker; automatic loading of Java class libs for spreadsheet I/O on Linux has been disabled in that version.

FYI, investigation on my Linux boxes showed it is the search for Java class libs from LibreOffice installation that apparently makes for a long delay (~5 seconds on my core i5 laptop).

Strikingly it doesn't take that much time on Windows (a mere 0.8 seconds for all class libs incl. LibreOffice).

I'll investigate later (no time now) to see if there's a better solution for Linux. So this automatic loading of Java class libs may return in a future io pkg release.

OTOH, as io-2.2.0 now has full read/write support for OOXML and ODS w/o external support SW (albeit a bit slow for ODS) and other file formats will hopefully follow, the need for Java class libs will become less and less stringent anyway.

Philip Nienhuis <philipnienhuis>
Project Member
Sat 05 Apr 2014 06:54:50 PM UTC, comment #8:

Thanks (at least one other user where that mechanism is confirmed to work).

Any suggestion on how to proceed?
I'm not sure if much automation is the right thing for the average Linux user, and perhaps this is too much behind-ones-back automation.

E.g., the operation of that environment variable could be reversed on Linux (more like "opt in" rather than "opt out").

Philip Nienhuis <philipnienhuis>
Project Member
Sat 05 Apr 2014 04:12:49 PM UTC, comment #7:

Setting OCTAVE_IO_JAVALIBS to `no` indeed fixes this:

maybe I'm just spoiled for never having to deal much with Java...

Carnë Draug <carandraug>
Project Member
Sat 05 Apr 2014 05:32:42 AM UTC, comment #6:

Oops, sorry, make that 'comment #3'

Philip Nienhuis <philipnienhuis>
Project Member
Sat 05 Apr 2014 05:31:34 AM UTC, comment #5:

Pls also try setting the env var to "no" as I outlined in comment #1. Can you report back then, please?

Maybe for Linux systems the search should be skipped; for the on average more helpless Windows users :-) it could be retained

Philip Nienhuis <philipnienhuis>
Project Member
Fri 04 Apr 2014 04:29:20 PM UTC, comment #4:

For me it takes a lot of time as well

I don't have any of java libraries required by the IO package though.

Carnë Draug <carandraug>
Project Member
Fri 04 Apr 2014 03:43:52 PM UTC, comment #3:

This may be due to the way the io pkg tries to automatically find Java spreadsheet class libs.

PKG_ADD (invoked when you load io) starts a function that searches in a few predefined places + subdirs (max 2 subdir levels deep), see below. So if you have many subdirs and files there, searches may take some time.

Hint: if you do not need spreadsheet I/O, try to set an environment flag:

export OCTAVE_IO_JAVALIBS=no

before starting Octave, then PKG_ADD will skip searching.

FYI, the places where PKG_ADD searches on Linux systems are (for "regular" spreadsheet I/O java class libs like Apache POI):
/usr/share/java/
~/java/
+ a subdir conveyed using environment var "OCTAVE_IO_JAVALIBS" (I suppose you didn't set that)

... and for OpenOffice /LibreOffice:
/usr/lib/<bnam>
/opt/<bnam>
where <bnam> is "OpenOffice.org", "LibreOffice" or "ooo", in that order.

So if you have exceptionally many files in one or more of those places PKG_ADD's search can take a while.

On all Linux systems I peeked in, /usr/share/java usually didn't have more than some tens of files so a search there should go fairly rapidly. Same holds for the other searched subdirs mentioned above.

Philip Nienhuis <philipnienhuis>
Project Member
Fri 04 Apr 2014 12:31:53 PM UTC, comment #2:

I don't doubt that it is taking a long time to load for you, but that is not the way to measure CPU time. The cputime function returns the number of CPU seconds since starting Octave. So you need to subtract the value returned from the first call:

John W. Eaton <jwe>
Project Administrator
Fri 04 Apr 2014 11:55:18 AM UTC, comment #1:

Seems to be related to java.
With java.opts memory usage can be controlled.

Stil io package takes long to load:

octave:2> cputime;pkg load io;cputime
ans = 34.632

When I recompile with --disable-java io package loads quickly.

Anonymous
Fri 04 Apr 2014 07:18:40 AM UTC, original submission:

if I launch the command-line version of octave 3.8.1, and then enter "pkg load io", this takes considerable time and, watching the process list with top, I see that the memory footprint (column VIRT in top) immediately goes to 33.6Gb. Is this normal?

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by mtmiller (Updated the item)
  • -unavailable- added by jwe (Posted a comment)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only project members can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 4 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Thu 17 Apr 2014 07:25:17 PM UTCphilipnienhuisStatusNeed Info=>Fixed
      Open/ClosedOpen=>Closed
    Sun 13 Apr 2014 02:55:58 PM UTCmtmillerCategoryInterpreter=>Octave Forge Package
      StatusNone=>Need Info

    Back to the top


    Powered by Savane 3.1-cleanup1