bugGNU Octave - Bugs: bug #39387, With large cell-arrays of numeric...

 
 

bug #39387: With large cell-arrays of numeric scalars, cell2mat is slow and memory-hungry.

Submitted by:  Olaf Till <i7tiol>
Submitted on:  Mon 01 Jul 2013 05:15:42 PM UTC  
 
Category: Octave FunctionSeverity: 3 - Normal
Priority: 5 - NormalItem Group: Performance
Status: FixedAssigned to: None
Originator Name: Olaf TillOpen/Closed: Closed
Release: devOperating System: GNU/Linux

Add a New Comment (Rich MarkupRich Markup):
   

You are not logged in

Please log in, so followups can be emailed to you.

 

Thu 18 Jul 2013 07:10:29 PM UTC, comment #2:

Great patches! I applied them to the development branch. I also re-vamped the input validation in another changeset (http://hg.savannah.gnu.org/hgweb/octave/rev/19b7c7412a63) which got me another ~25% savings.

Rik <rik5>
Project Administrator
Mon 01 Jul 2013 05:53:47 PM UTC, comment #1:

Sorry, I made a mistake in a test in cellfun, leading to a test failure. Corrected patch attached ("cellfun-isnumeric-2.changeset").

(file #28474)

Olaf Till <i7tiol>
Mon 01 Jul 2013 05:15:42 PM UTC, original submission:

The patches will be attached as soon as I have the bug number for the commit messages.

Slowness of cell2mat:

The latter command produced signes of thrashing due to memory exhaustion (severe disk access, slowing down of other applications).

(Same for reversed dimensions:

)

Slowness is partially due to a call, during argument checking:

which is made although there is no specialized method "isnumeric" in cellfun.

The rest of slowness and the thrashing seem to be due to the
cellfun("cat", ...) calls, which are necessary for cell-arrays
containing non-scalar elements. The code strives to minimize the number of cat() calls (within cellfun() calls) but seemingly has the sort order of the dimensions mistaken, it should be descending (which was the intention also according to the comments in the code). After correcting this, there is no thrashing (or less?) and the function is faster:

and with reversed dimensions

The attached cell2mat changeset makes the above correction and
furthermore introduces a special case for scalar-only elements, where one single cat() call can be used. This special case shortens duration from (here already with the cellfun changeset (see below), so both are faster than before):

to

I'd suggest applying this changeset to the stable branch.

The attached cellfun changeset adds the "isnumeric" special case to cellfun. With this:

(was 30 s before, see above), which adds to the speed enhancement of cell2mat.

I'd suggest applying this changeset to the default branch.

Olaf

Olaf Till <i7tiol>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

Attached Files
file #28474:  cellfun-isnumeric-2.changeset added by i7tiol (2kB - application/octet-stream)
file #28472:  cell2mat.changeset added by i7tiol (2kB - application/octet-stream)
file #28473:  cellfun-isnumeric.changeset added by i7tiol (2kB - application/octet-stream)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by rik5 (Posted a comment)
  • -unavailable- added by i7tiol (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only project members can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 6 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Thu 18 Jul 2013 07:10:29 PM UTCrik5StatusNone=>Fixed
      Open/ClosedOpen=>Closed
      Summary(Patches attached.) With large cell-arrays of numeric scalars, cell2mat is slow and memory-hungry.=>With large cell-arrays of numeric scalars, cell2mat is slow and memory-hungry.
    Mon 01 Jul 2013 05:53:47 PM UTCi7tiolAttached File-=>Added cellfun-isnumeric-2.changeset, #28474
    Mon 01 Jul 2013 05:31:29 PM UTCi7tiolAttached File-=>Added cell2mat.changeset, #28472
      Attached File-=>Added cellfun-isnumeric.changeset, #28473

    Back to the top


    Powered by Savane 3.1-cleanup