bugGNU Octave - Bugs: bug #40712, Running into index errors with...

 
 

bug #40712: Running into index errors with large, 64k square sparse matrices

Submitted by:  Paul Jakma <paul>
Submitted on:  Tue Nov 26 16:56:38 2013  
 
Category: LibrariesSeverity: 3 - Normal
Priority: 5 - NormalItem Group: Performance
Status: ConfirmedAssigned to: None
Originator Name: Open/Closed: Open
Release: devOperating System: GNU/Linux

Add a New Comment (Rich MarkupRich Markup):
   

You are not logged in

Please log in, so followups can be emailed to you.

 

Wed Nov 23 23:09:40 2016, comment #2:

These issues are still present in Octave 4.2.0, but I think they might all be "works for me" or "duplicate":

The first issue in comment #0 is ment to expand to a full matrix, since the 0 entries need to become 0**-0.5 = Inf (similar to bug #36562).

The second issue in comment #0 sounds like a duplicate of bug #40341 (Logical indexing into sparse matrices causes OOM) to me.

Hartmut <hardy>
Sun Mar 22 14:59:34 2015, comment #1:

Thanks for your bug report. I can confirm what you are seeing with the current development version of Octave. I can also confirm that building Octave with --enable-64 allows your examples to work. So that might be another viable path to pursue for working with datasets of the size you are talking about, sparse or not.

I am not familiar enough with the sparse implementation to know if this is a bug that deserves fixing so that it works in a default Octave built without 64-bit indexes.

Mike Miller <mtmiller>
Project Administrator
Tue Nov 26 16:56:38 2013, original submission:

I have a 64k × 64 matrix, sparse matrix:

spstats(D)
ans =
Compressed Column Sparse (rows = 1, cols = 64000, nnz = 15892 [25%])
...

I wanted to calculate D^-0.5, however it gives this error:

D = D**-0.5

error: memory exhausted or requested size too large for range of Octave's index type -- trying to return to prompt

It was suggested to me to on #octave use:

D(D>0) .^= -0.5

However, this gives the same error. In the end, the following workaround was found, which does work:

Ds = diag(D);
Ds(Ds > 0) .^= -0.5;
Ds = diag(Ds);

It would be good though if the first 2 ways above worked too.

While my matrices at the moment are 64k in size, they are likely to grow. They are actually large graphs, in a 32bit node space. The 64k size will grow and grow over time. Octave has been extremely useful in helping to analyse these graphs. It'd be very useful to me if Octave's index type could support operations on them.

Paul Jakma <paul>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by hardy (Posted a comment)
  • -unavailable- added by mtmiller (Updated the item)
  • -unavailable- added by paul (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only project members can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 4 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Sun Mar 22 14:59:34 2015mtmillerCategoryOctave Function=>Libraries
    Sun Jan 19 19:14:44 2014mtmillerCategoryNone=>Octave Function
      StatusNone=>Confirmed
      Release3.6.3=>dev

    Back to the top


    Powered by Savane 3.1-cleanup