bugGNU Octave - Bugs: bug #64017, axis limits can cause excessive...

 
 

bug #64017: axis limits can cause excessive memory usage for minor ticks including OOM crash

Submitter:  Martin Castillo <castilma>
Submitted:  Sat 08 Apr 2023 01:45:57 PM UTC
   
 
Category:  Plotting Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Performance
Status:  Fixed Assigned to:  None
Originator Name:  Martin Castillo Open/Closed:  * Closed
Release:  * 8.1.0 Operating System:  * GNU/Linux
Fixed Release:  8.3.0 Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Tue 18 Apr 2023 06:32:56 PM UTC, comment #7: 

I checked in the change on the stable branch here: http://hg.savannah.gnu.org/hgweb/octave/rev/1824e0ee4088.

Marking as Fixed and closing report.

Rik <rik5>
Group administrator
Tue 11 Apr 2023 04:37:44 PM UTC, comment #6: 

Turns out this is a problem for both Octave and Matlab.  Attached is a very minimal working example, and also shown below.


h = plot (1:10, zeros (1, 10), 'o');
%% Left-hand axes limit is very far below first tick at 1.
xlim ([-1e7, 10]);
%% Setting xtick is slow because number of minor ticks is calculated from
%% difference between xtick(2) - xtick(1) and then applied to the range from
%% axis_limit_low to xtick(1).  In the example given, this is roughly 50e6
%% elements of type double or 400 MB of memory.
set (gca, 'xtick', 1:10);


The problem code is in graphics.cc:calc_ticks_and_lims()


  // minor ticks between, above, and below min and max ticks
  int n = (is_logscale ? 8 : 4);
  double mult_below = (is_logscale ? tmp_ticks(1) / tmp_ticks(0) : 1);
  double mult_above = (is_logscale ? tmp_ticks(n_ticks-1) / tmp_ticks(n_ticks-2)
                       : 1);

  double d_below = (tmp_ticks(1) - tmp_ticks(0)) / mult_below / (n+1);
  int n_below = static_cast<int> (std::floor ((tmp_ticks(0)-lo_lim) / d_below));
  if (n_below < 0)
    n_below = 0;
  int n_between = n * (n_ticks - 1);
  double d_above = (tmp_ticks(n_ticks-1) - tmp_ticks(n_ticks-2)) * mult_above
                   / (n+1);
  int n_above = static_cast<int> (std::floor ((hi_lim-tmp_ticks(n_ticks-1))
                                  / d_above));
  if (n_above < 0)
    n_above = 0;

  Matrix tmp_mticks (1, n_below + n_between + n_above);


The problematic statement is the last one where a Matrix of minor tick values is created which can be very, very large.

For both Octave and Matlab, displaying these minorticks is impossible as they all overlap at the resolution of an ordinary monitor so this is simply an excessive waste of memory.

Attached patch limits the number of minor ticks below or above the actual range of ticks to 1000.  This seems fine to me,



(file #54601)

Rik <rik5>
Group administrator
Mon 10 Apr 2023 08:27:37 PM UTC, comment #5: 

This is what I see in my profiler (attached).

Dmitri.
--



Dmitri A. Sergatskov <dasergatskov>
Mon 10 Apr 2023 08:13:10 PM UTC, comment #4: 

I extracted a more minimal example and attached tst_xtick_hang.m.

The issue is with a statement


set (gca, 'xticks', ticks);


This seems to cause some near infinite loops in C++ in the function calc_ticks_and_lims().

(file #54596)

Rik <rik5>
Group administrator
Mon 10 Apr 2023 06:57:33 PM UTC, comment #3: 

On my machine the second line makes octave to use 8.5 Gb of ram
and finishes in about 7.5 seconds. The profile run is attached.
I also added "whos" in hist/bar/__bar__ but do not see obviously large variables.

Dmitri.
--



Dmitri A. Sergatskov <dasergatskov>
Mon 10 Apr 2023 06:21:58 PM UTC, comment #2: 

It smells like another slice optimization gone wrong.

Dmitri.
--

Dmitri A. Sergatskov <dasergatskov>
Mon 10 Apr 2023 05:53:00 PM UTC, comment #1: 

Confirmed.

This is really weird because the number of bins isn't particularly large (10, 11, or 12).  The first code takes 2.6 seconds on my machine.  The second caused enormous memory usage and I ended up killing the process.

I can confirm that asking for the inputs is basically instantaneous, but when I go to plot using bar() Octave hangs.

Rik <rik5>
Group administrator
Sat 08 Apr 2023 01:45:57 PM UTC, original submission:  

Consider the following three commands:

octave:26> hist(1:5,10.^(1:10))
Aoctave:27> hist(1:5,10.^(1:11))
error: out of memory or dimension too large for Octave's index type
error: called from
    __bar__ at line 237 column 16
    bar at line 117 column 18
    hist at line 249 column 5
octave:28> hist(1:5,10.^(1:12))


The first one takes about two seconds on my machine.
The second is faster and throws that error.
The third is faster and throws no error, though it has more work to do (one more bin).

After the first command, octave uses 1GB residential memory.
After the second and third it drops down to 180MB.

You may want to disable swap (sudo swapoff -a) before executing the second one. Otherwise your system becomes unresponsive for a while.

They all run very fast an show no error when output is expected:

[a,b] = hist(1:5, 10.^(1:11))

So I think the problem must be with the plotting subsystem.

Martin Castillo <castilma>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #54602:  tst_minortick.m added by rik5 (416B - text/x-matlab)
file #54601:  bug64017.cset added by rik5 (2KiB - application/octet-stream)
file #54597:  xtick_prof.png added by dasergatskov (219KiB - image/png)
file #54596:  tst_xtick_hang.m added by rik5 (990B - text/x-matlab)
file #54595:  hist_das.png added by dasergatskov (119KiB - image/png)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by dasergatskov (Posted a comment)
  • -email is unavailable- added by rik5 (Posted a comment)
  • -email is unavailable- added by castilma (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 13 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2023-04-18 rik5 StatusPatch Submitted Fixed
        Open/ClosedOpen Closed
        Fixed ReleaseNone 8.3.0
    2023-04-11 rik5 Attached File- Added tst_minortick.m, #54602
    2023-04-11 rik5 StatusConfirmed Patch Submitted
        Summarypatch object with set (gca, 'xtick', ticks) causes near hang or OOM axis limits can cause excessive memory usage for minor ticks including OOM crash
    2023-04-11 rik5 Attached File- Added bug64017.cset, #54601
    2023-04-10 dasergatskov Attached File- Added xtick_prof.png, #54597
    2023-04-10 rik5 Attached File- Added tst_xtick_hang.m, #54596
        Summaryplotting with hist() results in unnecessary big (/failing) memory allocation for some inputs. patch object with set (gca, 'xtick', ticks) causes near hang or OOM
    2023-04-10 dasergatskov Attached File- Added hist_das.png, #54595
    2023-04-10 rik5 Item GroupNone Performance
        StatusNone Confirmed

    Back to the top

    Powered by Savane 3.13-0329.
    Corresponding source code