bugGNU Octave - Bugs: bug #39378, randperm cannot handle large...

 
 

bug #39378: randperm cannot handle large arguments

Submitter:  Marco Caliari <caliari>
Submitted:  Mon 01 Jul 2013 07:28:51 AM UTC
   
 
Category:  Octave Function Severity:  3 - Normal
Priority:  5 - Normal Item Group:  None
Status:  Fixed Assigned to:  None
Originator Name:  Open/Closed:  * Closed
Release:  * 3.6.2 Operating System:  * GNU/Linux
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Tue 15 Oct 2013 06:53:27 PM UTC, comment #8: 

I have finally pushed my fix:

http://hg.savannah.gnu.org/hgweb/octave/rev/0b7f5c56f853

This should be part of the upcoming release.

Jordi Gutiérrez Hermoso <jordigh>
Group Member
Wed 17 Jul 2013 05:20:32 PM UTC, comment #7: 

At the time I wrote my original comments, I was going off memory without reading the actual source code. Now that I look at it again, I realise that a slightly more sophisticated fix is required, first see if m will fit into an octave_idx_type, and if it doesn't then force a short shuffle.

It's a bit more than a one-liner right now, and at the moment I can't dedicate any time to this possibly easy problem.

Jordi Gutiérrez Hermoso <jordigh>
Group Member
Wed 17 Jul 2013 03:43:41 PM UTC, comment #6: 

Jordi, can you apply this one-liner fix of yours or is there more testing to be done?

Rik <rik5>
Group administrator
Wed 03 Jul 2013 02:01:50 PM UTC, comment #5: 

Perfect.

Marco

Marco Caliari <caliari>
Group Member
Wed 03 Jul 2013 01:26:11 PM UTC, comment #4: 

No, I meant double(n) > ..., rather. The problem is that if the full size of the vector is too big, then there's no hope that the "long" shuffle can work, because then line 1149 fails:

    http://hg.savannah.gnu.org/hgweb/octave/file/76bd90f6ba65/libinterp/corefcn/rand.cc#l1147

In this case, we must do a short shuffle.

Jordi Gutiérrez Hermoso <jordigh>
Group Member
Wed 03 Jul 2013 08:25:28 AM UTC, comment #3: 

Probably you mean


double(m) < double...


Marco

Marco Caliari <caliari>
Group Member
Tue 02 Jul 2013 07:40:12 PM UTC, comment #2: 

I had some reason for capping the size of M. My reasoning was that if M was big enough, you might as well just allocate the full vector for speed. Perhaps a more sophisticated approach would be to do


bool short_shuffle = (m < n/5 && m < 1e5) || double(m) > double(std::numeric_limits<octave_idx_type>::max());


?

Jordi Gutiérrez Hermoso <jordigh>
Group Member
Tue 02 Jul 2013 02:02:49 PM UTC, comment #1: 

The workaround is to change the "quick and dirty" test in rand.cc into


bool short_shuffle = m < n/5;


Marco

Marco Caliari <caliari>
Group Member
Mon 01 Jul 2013 07:28:51 AM UTC, original submission:  

Dear all,

the following


N=30000^2;,M=100000;,randperm(N,M);


gives me the "memory exhausted or requested size too large for range of Octave's index type" error. I think it should work since:

1) N < intmax
2) M < N/5 and so O(M) memory should be used (according to help)
3) Matlab works with such an example

Best regards,

Marco

Marco Caliari <caliari>
Group Member

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by rik5 (Posted a comment)
  • -email is unavailable- added by jordigh (Updated the item)
  • -email is unavailable- added by caliari (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 4 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2013-10-15 jordigh StatusConfirmed Fixed
        Open/ClosedOpen Closed
    2013-07-17 jordigh StatusNone Confirmed
    2013-07-02 jordigh SummaryPossible bug in randperm randperm cannot handle large arguments

    Back to the top

    Powered by Savane 3.13-d3ae.
    Corresponding source code