bugGNU Octave - Bugs: bug #40341, Logical indexing into sparse...

 
 

bug #40341: Logical indexing into sparse matrices causes OOM

Submitted by:  David Spies <dspyz>
Submitted on:  Mon 21 Oct 2013 11:08:04 PM UTC  
 
Category: Octave FunctionSeverity: 3 - Normal
Priority: 5 - NormalItem Group: Incorrect Result
Status: ConfirmedAssigned to: None
Originator Name: Open/Closed: Open
Release: devOperating System: GNU/Linux

Add a New Comment (Rich MarkupRich Markup):
   

You are not logged in

Please log in, so followups can be emailed to you.

 

(Jump to the original submission Jump to the original submission)

Fri 07 Feb 2014 09:47:30 PM UTC, comment #9:

Oh no,
v = nonzeros(A .* B);
doesn't quite mean the same thing as
v = A(B)
since it ignores the zeros

David Spies <dspyz>
Fri 07 Feb 2014 06:26:18 PM UTC, comment #8:

Well, that's definitely a clever work around. Good stuff.

Jordi Gutiérrez Hermoso <jordigh>
Project Administrator
Fri 07 Feb 2014 06:16:39 PM UTC, comment #7:

That is an entirely valid and reasonable request, however I'm afraid I can't ask my supervisor for money for this right now.

I do think I've found a workaround though (at least for numeric matrices).

To say A(B) = 0
I can instead use A = A - A .* B;

And to mean A(B) = C(B);

I can instead use A = A - A .* B + C .* B;

And for v = A(B), I can say v = nonzeros(A .* B);

And for A(B) = v, I can say
[i,j] = find(B);
[m,n] = size(B);
A = A - A .* B + sparse(i,j,v,m,n);

David Spies <dspyz>
Fri 07 Feb 2014 05:05:46 PM UTC, comment #6:

Oh, right, stupid me.

There is no workaround other than looping through the i and j vectors.

You seem to really want this. Would you be able to tip me a little bit of money or ask your employer to do so if I spent a few hours or days trying to fix this about Octave? This problem doesn't affect me, and it doesn't affect most Octave users, and it would take a lot of effort to fix. Since you're the only person I know who is affected but you're apparently unable to fix Octave yourself, it would got a long way to motivate me to fix this if there was a little bit of money involved.

If not, you might wait until David Bateman tries to fix this. He might have other motivations to do so. Or perhaps another champion will arise from the shadows to squash this bug.

Jordi Gutiérrez Hermoso <jordigh>
Project Administrator
Fri 07 Feb 2014 04:21:04 PM UTC, comment #5:

That's not the same thing.

a(i,j) refers to all elements whose row is in the set i and whose column is in the set j.

But I only want the set of things whose (row,column) are among the set of pairs (i,j)

David Spies <dspyz>
Fri 07 Feb 2014 02:31:46 PM UTC, comment #4:

A simple workaround is to use find:

Jordi Gutiérrez Hermoso <jordigh>
Project Administrator
Fri 07 Feb 2014 08:44:09 AM UTC, comment #3:

Can someone take another look at this? It's really driving me nuts. Is there a workaround I can use?

David Spies <dspyz>
Tue 22 Oct 2013 06:27:27 PM UTC, comment #2:

This one is documented in the projects file

http://wiki.octave.org/Projects#Sparse_Matrices

What is happening now in your case is that the logical "b" is being converted to a single argument numeric index for the "true" values of the sparse logical matrix. This is identical to what happens for normal logical matrix indexing, but in the case of sparse matrices the values of the numerically index can depasse intmax().

The solution is the create a new idx_vector class particularly for sparse logical indexing that stores (row,column) pairs. If used in its default form it would act identically to idx_vector::idx_vector_rep, but the Sparse indexed assignment could then special case it.

There is no call to numel() anywhere here

David Bateman <dbateman>
Project Member
Tue 22 Oct 2013 12:50:46 AM UTC, comment #1:

Ah, this is one that you won't be able to fix by avoiding calls to numel, Mr Bateman...

Jordi Gutiérrez Hermoso <jordigh>
Project Administrator
Mon 21 Oct 2013 11:08:04 PM UTC, original submission:

>> a = sprand(100000,100000,0.0001);
>> b = logical(sprand(100000,100000,0.0001));
>> a(b) = 0;

error: out of memory or dimension too large for Octave's index type

Note:
This bug also occurs in 3.6.4

David Spies <dspyz>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by dbateman (Posted a comment)
  • -unavailable- added by jordigh (Posted a comment)
  • -unavailable- added by dspyz (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only project members can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follows 1 latest change.

    Date Changed By Updated Field Previous Value => Replaced By
    Tue 22 Oct 2013 12:50:46 AM UTCjordighStatusNone=>Confirmed

    Back to the top


    Powered by Savane 3.1-cleanup