bugGNU Octave - Bugs: bug #40357, OOM when converting from sparse...

 
 

bug #40357: OOM when converting from sparse double to sparse logical

Submitter:  David Spies <dspyz>
Submitted:  Wed 23 Oct 2013 06:43:22 PM UTC
   
 
Category:  Octave Function Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Incorrect Result
Status:  Need Info Assigned to:  None
Originator Name:  Open/Closed:  * Open
Release:  * dev Operating System:  * GNU/Linux
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Sat 13 Jan 2024 02:41:38 AM UTC, comment #7: 

And on Raspberry Pi 4 (8G mem + 16G swap) / Fedora 39:

octave:1> __octave_config_info__.ENABLE_64
ans = 1
octave:2> x = logical(sparse(100000,100000));
octave:3> y = sparse(100000,100000);
octave:4> ii = full(logical(sprand(100000,1,0.3)));
octave:5> x(ii,:) = y(ii,:);
Killed

From system log:


Jan 12 21:34:38 raspi4 kernel: oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/user.slice/user-1001.slice/session-2.scope,task=octave-gui,pid=47967,uid=1001
Jan 12 21:34:38 raspi4 kernel: Out of memory: Killed process 47967 (octave-gui) total-vm:24233516kB, anon-rss:7204264kB, file-rss:9400kB, shmem-rss:0kB, UID:1001 pgtables:46276kB oom_score_adj:0


Dmitri.
--

Dmitri A. Sergatskov <dasergatskov>
Sat 13 Jan 2024 12:33:36 AM UTC, comment #6: 

On 8 GB machine

octave:1> __octave_config_info__.ENABLE_64
ans = 1
octave:2> x = logical(sparse(100000,100000));
octave:3> y = sparse(100000,100000);
octave:4> ii = full(logical(sprand(100000,1,0.3)));
octave:5> x(ii,:) = y(ii,:);
error: out of memory or dimension too large for Octave's index type
octave:6> memory
Octave is running on x86_64-pc-linux-gnu
System    RAM:   8089480 kiB,  swap:   4194300 kiB
Octave    RAM:     80536 kiB,  virt:    597508 kiB
Free      RAM:   1426300 kiB,  swap:   4194044 kiB
Available RAM:   7254052 kiB, total:  11448096 kiB
octave:7> version -hgid
ans = 1c40194c7b1f


Dmitri.
--

Dmitri A. Sergatskov <dasergatskov>
Fri 12 Jan 2024 11:40:15 PM UTC, comment #5: 

I'm fortunate enough to have almost exclusively underpowered systems. I'll do a few tests and see what happens.

Nicholas Jankowski <nrjank>
Group Member
Fri 12 Jan 2024 10:30:33 PM UTC, comment #4: 

@nrjank: This works for me now (I'm testing on development branch or 10.0).  It might be because I have a 64-bit octave_idx_type.  It might also be because I have 32 GB of RAM.  If you also don't have problems then I would be inclined to close this.

Rik <rik5>
Group administrator
Thu 11 Nov 2021 09:29:13 PM UTC, comment #3: 

behavior is still present in Octave 6.4.0.  not certain if it's a duplicate, if internally it's the indexing causing the the 'full' expansion and OOM event, or if it's something else regarding the type change. Note that changing y to also be logical, then:


x = logical(sparse(100000,100000));
y = logical(sparse(100000,100000));
i = full(logical(sprand(100000,1,0.3)));
x(i,:) = y(i,:);


results in no memory issues. (although perhaps octave is just being efficient resulting in it sidestepping an indexing problem when the types are the same?).  perhaps there's a better test to determine if this is a duplicate of the indexing issue, or something else.

Nicholas Jankowski <nrjank>
Group Member
Fri 25 Nov 2016 09:46:40 PM UTC, comment #2: 

I think this might be a DUPLICATE of bug #40341 (Logical indexing into sparse matrices causes OOM).

It is also already mentioned as an open todo for the Octave sparse matrix implementation: http://wiki.octave.org/Projects#Sparse_Matrices (Sparse logical indexing...)

Hartmut <hardy>
Tue 22 Nov 2016 10:37:45 PM UTC, comment #1: 

This behavior is still present in Octave 4.2.0.

Hartmut <hardy>
Wed 23 Oct 2013 06:43:22 PM UTC, original submission:  


>> x = logical(sparse(100000,100000));
>> y = sparse(100000,100000);
>> i = full(logical(sprand(100000,1,0.3)));
>> x(i,:) = y(i,:);

error: out of memory or dimension too large for Octave's index type

David Spies <dspyz>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by dasergatskov (Posted a comment)
  • -email is unavailable- added by rik5 (Posted a comment)
  • -email is unavailable- added by nrjank (Posted a comment)
  • -email is unavailable- added by hardy (Posted a comment)
  • -email is unavailable- added by mtmiller (Updated the item)
  • -email is unavailable- added by dspyz (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 2 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2021-11-11 nrjank StatusConfirmed Need Info
    2014-01-19 mtmiller StatusNone Confirmed

    Back to the top

    Powered by Savane 3.13-d3ae.
    Corresponding source code