bugGNU Octave - Bugs: bug #47468, ones(), zeros(), pi(), e() are...

 
 

bug #47468: ones(), zeros(), pi(), e() are able to create a row vector with number of elements above sizemax

Submitter:  Carnë Draug <carandraug>
Submitted:  Sat 19 Mar 2016 07:04:24 PM UTC
   
 
Category:  Libraries Severity:  3 - Normal
Priority:  5 - Normal Item Group:  None
Status:  Confirmed Assigned to:  None
Originator Name:  Open/Closed:  * Open
Release:  * dev Operating System:  * GNU/Linux
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Mon 04 Nov 2019 04:22:23 PM UTC, comment #5: 

Confirmed.  This is still an issue.  It appears that the Range constructor should probably check to see whether numel() is exceeded.

Rik <rik5>
Group administrator
Sun 27 Oct 2019 05:05:33 PM UTC, comment #4: 

This is quite an old bug report, but I made some progress on it recently.  The issue is that Octave had an optimization for ones() which allowed it to store row vectors as Range objects.  The memory savings could be significant, a fixed 24 bytes for a Range, versus a variable 8 x numel(x) for any other sort of object.

I made the optimization to include zeros(), pi(), and e() as well in this cset https://hg.savannah.gnu.org/hgweb/octave/rev/e78b6e7f743c.

Now, of course, all of these functions have the same problem of being able to create an overly large array.  I plan to ask on the Octave Maintainers mailing list about how we want to handle this.

Rik <rik5>
Group administrator
Fri 30 Dec 2016 10:52:51 AM UTC, comment #3: 

I can confirm this problem with octave 3.8.2 (Ubuntu Linux 12.04) too for zeros function.

When I do a fresh start of octave with experimental forced GUI I can generate


-----------------------
test = zeros(18059);
-----------------------


without errors.

When I try to add some other variables to that, lets say


------------------------
n=0;
while(1)
  n++;
  dump{n} = zeros();
endwhile
------------------------


it is even possible to add a few arrays with small size (~10 - 50 square arrays).

.

Then I clear everything and repeat the steps but

even the 18059*18059 zeros aren't working.

Add some point even arrays with



test = zeros(floor(sqrt(sizemax/8.0)-1.0));


won't work.


Is there no possiblity to check how much unfragmented memory is still available for arrays in octave by a



system('some script')



My System Specs (lshw -short) and Linux distro is...


H/W-Pfad        Gerät      Klasse         Beschreibung
=======================================================
                            system         HP ProBook 450 G1 (F7X73ES#ABD)
/0                          bus            1942
/0/c                        memory         64KiB BIOS
/0/0                        processor      Intel(R) Core(TM) i5-4200M CPU @ 2.50GHz
/0/0/2                      memory         32KiB L1 Cache
/0/0/3                      memory         256KiB L2 Cache
/0/0/4                      memory         3MiB L3 Cache
/0/0/2.1                    processor      Logische CPU
/0/0/2.2                    processor      Logische CPU
/0/0/2.3                    processor      Logische CPU
/0/0/2.4                    processor      Logische CPU
/0/0/2.5                    processor      Logische CPU
/0/0/2.6                    processor      Logische CPU
/0/0/2.7                    processor      Logische CPU
/0/0/2.8                    processor      Logische CPU
/0/0/2.9                    processor      Logische CPU
/0/0/2.a                    processor      Logische CPU
/0/0/2.b                    processor      Logische CPU
/0/0/2.c                    processor      Logische CPU
/0/0/2.d                    processor      Logische CPU
/0/0/2.e                    processor      Logische CPU
/0/0/2.f                    processor      Logische CPU
/0/0/2.10                   processor      Logische CPU
/0/1                        memory         32KiB L1 Cache
/0/6                        memory         16GiB Systemspeicher
/0/6/0                      memory         8GiB SODIMM DDR3 Synchron 1600 MHz (0,6 ns)
/0/6/1                      memory         8GiB SODIMM DDR3 Synchron 1600 MHz (0,6 ns)
/0/100                      bridge         Xeon E3-1200 v3/4th Gen Core Processor DRAM Controller
/0/100/1                    bridge         Xeon E3-1200 v3/4th Gen Core Processor PCI Express x16 Controller
/0/100/1/0                  generic        Illegal Vendor ID
/0/100/2                    display        4th Gen Core Processor Integrated Graphics Controller
/0/100/3                    multimedia     Xeon E3-1200 v3/4th Gen Core Processor HD Audio Controller
/0/100/14                   bus            8 Series/C220 Series Chipset Family USB xHCI
/0/100/16                   communication  8 Series/C220 Series Chipset Family MEI Controller #1
/0/100/1a                   bus            8 Series/C220 Series Chipset Family USB EHCI #2
/0/100/1b                   multimedia     8 Series/C220 Series Chipset High Definition Audio Controller
/0/100/1c                   bridge         8 Series/C220 Series Chipset Family PCI Express Root Port #1
/0/100/1c.5                 bridge         8 Series/C220 Series Chipset Family PCI Express Root Port #6
/0/100/1c.5/0   eth0        network        RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller
/0/100/1c.6                 bridge         8 Series/C220 Series Chipset Family PCI Express Root Port #7
/0/100/1c.6/0   wlan0       network        Wireless 7260
/0/100/1c.7                 bridge         8 Series/C220 Series Chipset Family PCI Express Root Port #8
/0/100/1c.7/0               generic        Realtek Semiconductor Co., Ltd.
/0/100/1d                   bus            8 Series/C220 Series Chipset Family USB EHCI #1
/0/100/1f                   bridge         HM87 Express LPC Controller
/0/100/1f.2                 storage        8 Series/C220 Series Chipset Family 6-port SATA Controller 1 [AHCI m
/0/2            scsi4       storage
/0/2/0.0.0      /dev/sda    disk           500GB HGST HTS545050A7
/0/2/0.0.0/1    /dev/sda1   volume         461GiB EXT4-Laufwerk
/0/2/0.0.0/2    /dev/sda2   volume         3976MiB Extended partition
/0/2/0.0.0/2/5  /dev/sda5   volume         3976MiB Linux swap / Solaris partition
/0/3            scsi5       storage
/0/3/0.0.0      /dev/cdrom  disk           DVD-RAM UJ8DB
/1                          power          FP06047



Linux blanedbyuser 3.13.0-106-generic #153~precise1-Ubuntu SMP Tue Dec 6 16:14:59 UTC 2016 i686 i686 i386 GNU/Linux


best regards

Niko

Anonymous
Wed 30 Mar 2016 10:58:06 AM UTC, comment #2: 

I think Octave has a real indexing problem in large border cases.
For sparse matrices it depends whether the exceeding dimension was given as first or second argument. (see bug #45963)

Kai Torben Ohlhus <siko1056>
Group Member
Sat 19 Mar 2016 07:37:41 PM UTC, comment #1: 

Actually, I am probably on the border of my available memory so the error comes from there instead of input checking (for the case of zeros).

But the original issue remains that 'ones (1, sizemax ()+1)' is doing something odd (now that I think about it is also returning too quickly). It behaves as expected when I specify its type:


octave> uint64 (numel (ones (1, sizemax (), "uint8")))
ans = 2147483646
octave> uint64 (numel (ones (1, sizemax () +1, "uint8")))
error: out of memory or dimension too large for Octave's index type
octave> uint64 (numel (ones (sizemax (), 1, "uint8")))
ans = 2147483646
octave> uint64 (numel (ones (1, 1, sizemax (), "uint8")))
ans = 2147483646
octave> uint64 (numel (zeros (sizemax (), 1, "uint8")))
ans = 2147483646


Carnë Draug <carandraug>
Group Member
Sat 19 Mar 2016 07:04:24 PM UTC, original submission:  

The functions ones() is able to create an array with more elements than it should.

The problem is probably on input check. It is able to create an array with one element above sizemax (I will report a separate bug for the limit of sizemax which is one below the numeric limit of int32).


octave> uint64 (numel (ones (1, sizemax ())))
ans = 2147483646
octave> uint64 (numel (ones (1, sizemax () +1)))
ans = 2147483647
octave> uint64 (numel (ones (1, sizemax () +10)))
ans = 2147483647


Interestingly, this only works when sizemax is the number of columns:


octave> uint64 (numel (ones (sizemax () +1, 1)))
error: out of memory or dimension too large for Octave's index type
octave> uint64 (numel (ones (1, 1, sizemax () +1)))
error: out of memory or dimension too large for Octave's index type


zeros () is more sensitive the other way and even fails when creating an array with numel == sizemax:


octave> uint64 (numel (zeros (1, sizemax ())))
error: out of memory or dimension too large for Octave's index type
octave> uint64 (numel (zeros (sizemax (), 1)))
error: out of memory or dimension too large for Octave's index type


and even for less elements than sizemax():


octave> uint64 (numel (zeros (sizemax ()-1, 1)))
error: out of memory or dimension too large for Octave's index type


Carnë Draug <carandraug>
Group Member

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by rik5 (Posted a comment)
  • -email is unavailable- added by siko1056 (Posted a comment)
  • -email is unavailable- added by carandraug (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 2 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2019-11-04 rik5 StatusNone Confirmed
    2019-10-27 rik5 Summaryones() is able to create array with number of elements above sizemax ones(), zeros(), pi(), e() are able to create a row vector with number of elements above sizemax

    Back to the top

    Powered by Savane 3.13-f8d8.
    Corresponding source code