bugGNU Octave - Bugs: bug #53644, Possible Octave or Nouveau image...


bug #53644: Possible Octave or Nouveau image plot bug (not sure which) discovered via ASAN

Submitted by:  Dan Sebald <sebald>
Submitted on:  Fri 13 Apr 2018 06:28:23 PM UTC  
Category:  Plotting with OpenGL Severity:  3 - Normal
Priority:  5 - Normal Item Group:  None
Status:  Fixed Assigned to:  None
Originator Name:  Open/Closed:  Closed
Release:  dev Operating System:  GNU/Linux

Add a New Comment(Rich Markup)

You are not logged in

Please log in, so followups can be emailed to you.


( Jump to the original submission)

Thu 19 Apr 2018 04:13:21 PM UTC, comment #9:

I've done the full build and "make check" on a system with Nvidia video driver active and a system with Nouveau driver active. As far as graphics, both succeed (yay!).

It wouldn't surprise me if this issue may have caused problems on other systems and graphics cards, but in those situations the difference could have been that maybe Nouveau is using system memory for its implementation, hence ASAN catches an address-out-of-bounds error, whereas the Intel/Nvidia native graphics drivers use their internal memory for general calculations (since the companies know exactly what memory their own cards have). Just a guess. But in the latter situation there's no OS to catch problems internal to the graphics card so however it is programmed (right or wrong) is what happens.

Dan Sebald <sebald>
Mon 16 Apr 2018 05:06:29 AM UTC, comment #8:

Very useful. This fix also improves line smoothness in
FC 27 using Intel VGA controller

description: VGA compatible controller
product: Intel Corporation
vendor: Intel Corporation
physical id: 2
bus info: pci@0000:00:02.0
version: 06
width: 64 bits
clock: 33MHz
capabilities: pciexpress msi pm vga_controller bus_master cap_list rom
configuration: driver=i915 latency=0
resources: irq:126 memory:db000000-dbffffff memory:90000000-9fffffff ioport:f000(size=64) memory:c0000-dffff

Michael Godfrey <godfrey>
Project Member
Mon 16 Apr 2018 01:41:52 AM UTC, comment #7:

Oh boy, that was a cryptic bug. I checked in your changeset here: http://hg.savannah.gnu.org/hgweb/octave/rev/601cc3a063f5.

I don't use the nouveau driver, but I had occasionally seen segfaults when building the plots for the documentation as well. I hope this fixes it.

Closing report.

Rik <rik5>
Project Administrator
Sun 15 Apr 2018 09:23:20 AM UTC, comment #6:

Found it. Whew, that one was a "doozy" (using mid-20th century idiom). It's a bug in plain view, but until one realizes how it comes about, it's baffling.

So, the main clue is that the opengl_renderer::draw_axes routine fails in the same code line every time, but it isn't until the second invocation of "image()" that the ASAN abort occurs. Look at the following code and notice there are two instances of draw_axes_boxes()

Basically, there was nothing disabling any possibly-set glLineStipple in the draw_axes_boxes() routine such that on the second invocation of image() the stipple was set coming into this routine. The draw_axes_planes() routine uses GL_QUADS, but filled quads have a line option. I suppose having stipple on with those quads causes Nouveau to look for stipple information just outside the valid memory region for the stored quad. That is, in this ASAN output:

the [fa] is the out-of-range address access (00 means addressable) just after what is probably the quad stored in memory. One could say this may be a Nouveau issue because it should be able to address the situation when stipple is on, but as I see it we really aren't using such a scenario so I think we can avoid it. Plus, it makes good sense to have every instance of setting glLineStipple balanced with turning it off after line-drawing is done.

Attached is a patch that makes the ASAN dump go away, and it also makes the documentation figure failures go away on KDE/Plasma where I'm using Nouveau driver and not Nvidia driver.

(file #43931)

Dan Sebald <sebald>
Sat 14 Apr 2018 04:09:23 PM UTC, comment #5:

Regarding Comment #4, after testing fairly extensively last night, I don't think the multiple drawing of a newly created figure is the root cause of this issue. I'm going to guess that perhaps the multiple "rendering" comes about from the OS window manager. (I'm going to carefully use the word "rendering" as specifically referring to the OpenGL actions and "redraw" as referring to Octave running its plot code.) That is, some of these window managers get sort of fancy with what they are doing. For example, I think Cinnamon will render a new app window multiple times--small-to-big--as though the window is appearing out of the ether or something. It could be that Plasma is doing the same, but the window is always the same size, and the randomness of how many draws may come from the window manager drawing based on timing, e.g., every 150 ms for 1 s factoring in the rendering speed. So, it could be that the OS window manager creates the figure window and keeps rendering the thing while Octave is still populating the figure. First it is a blank window, i.e.,

xPlane: 1
yPlane: 1
zPlane: 0
xPlaneN: 0
yPlaneN: 0
zPlaneN: 1

and somewhere along the way it becomes a populated window, i.e.,

xPlane: 64.5
yPlane: 0.5
zPlane: 0
xPlaneN: 0.5
yPlaneN: 64.5
zPlaneN: 1
is2d: 1

In any case, what that multiple rendering does when first creating the Figure is it brings the root cause to the core because of the following description. It appears to be the first rendering after an "image" rendering is causing the ASAN abort. That is, if I first generate

>> figure

then separately call

>> image

there is no ASAN abort, but from that point forward anything redrawn (not necessarily an image) will cause an abort in the opengl_renderer::draw_axes_planes() rendering. It doesn't have to be another image(). And the failure is consistently at that same location I first described.

Now, it is only the image() routine that will setup the failure condition. I can run ploat(1:50), surface(), etc. and keep replotting...none of that rendering fails. It's only when I call image() then do following rendering, whether it is another Octave plot action, or resizing the window, etc.

So, the root cause lies in the image() rendering code, and it is difficult to find. I've practically eliminated the draw_image() OpenGL-aimed code and the issue still occurs. I wonder if maybe there is a colormap or some other ancillary issue unique to image plotting. Maybe within the OpenGL driver Octave is instructing a colormap that is slightly too big causing the heap out of bounds.

Dan Sebald <sebald>
Sat 14 Apr 2018 06:20:58 AM UTC, comment #4:

Can anyone think of a recent change to the graphics code that would cause multiple plotting prior to Figure data being all initialized?

Dan Sebald <sebald>
Sat 14 Apr 2018 06:09:06 AM UTC, comment #3:

The data all sort of makes sense to me in terms of the opengl_renderer::draw_axes_planes() routine. But something is not quite right about the initialization of the figure/graphics. If I call

>> figure; image

immediately after launching octave, I see:

ASAN aborts. And if I use

>> figure
>> image

I see:

i.e., success. Notice there isn't that extra rounds of calling this routine in the latter example.

I think that this is the timing issue bug that is fairly recent


Notice how in the failing first case I used "figure; image". In the successful latter case I used "figure" followed by "image". That time it took me to type "i m a g e <CR>" was easily enough of a delay to do effectively what Rik's doing with adding a delay. So, the above is a clue as to what is happening. It looks to me that the figure is created, perhaps only partially, while plotting is allowed to proceed. Some kind of recursion in the plotting action is going on until (maybe) some variable being set is stopping the recursion.

Reiterating, there should only be one call to opengl_renderer::draw_axes_planes() in drawing this simple image figure, not five calls to draw_axes_planes(). Notice how the first few passes are using a box of (0,0) to (1,1) (probably default upon creating the figure) while the last few have a box more in line with the image data (0.5,0.5) to (64.5,64.5). And the number of passes is random when I try this multiple times.

The same type of scenario occurs if I do a data plot like "figure; plot(1:50)", but that doesn't seem to cause a pointer outside of memory block so ASAN doesn't abort.

Dan Sebald <sebald>
Sat 14 Apr 2018 03:22:27 AM UTC, comment #2:

Not sure this is specific to images. Nonetheless, I'm writing notes here to keep track of things.

It seems that the ASAN consistently exits at "spo 4" in the following location in gl-render.cc:

The image() routine causes the above hunk of code to run three times (in pairs, i.e., the "on" is true, some drawing, then "on" is false) and on the third time that "glDisable (GL_POLYGON_OFFSET_FILL);" is where ASAN complains and exits.

That routine glDisable() is quite simple, it's basically disabling the polygon features or drawing a filled surface and an outlining or border. So I can't imagine that routine is the actual problem, just where ASAN recognizes the problem. Also, glPolygonOffset() is pretty innocuous, as it controls a little z-depth offset to bring the lines nearer and prevent the stitching phenomenon.

So, as is typical with these ASAN like programs, we look prior to the occurrence of the dump. I worked backward and put pre-process conditional around the following hunk of code

and the image() routine works fine, can be repeated, have different plotting methods in-between, etc. So, the next step is to look for anything peculiar about the memory buffer for the drawing of the axes in this code hunk.

Dan Sebald <sebald>
Fri 13 Apr 2018 06:32:36 PM UTC, comment #1:

Rik commented here:


"We could put an assert statement in the code temporarily to make sure that offset is in some reasonable range ([-5, +5])."

If it is temporary, for the sake of finding a bug, why not make the assert the exact range? If the image is shifted, say one pixel to the left/right or top/bottom, we'd want to know that. That would be the likely risk here; some kine of 0-start versus 1-start indexing.

Dan Sebald <sebald>
Fri 13 Apr 2018 06:28:23 PM UTC, original submission:

In Bug #53604 I've scene an ancillary issue with some OpenGL image plotting:

From my recollection reading other bugs in the track, Nouveau is known to have some issues. It's only on some KDE test platforms that Nouveau is run, whereas on my Mint/Cinnamon system the Nvidia driver is used. It could be that Nvidia driver is more forgiving, if there happened to be an Octave bug in using OpenGL, but I'm not sure what the issue is.

Dan Sebald <sebald>


(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:


Depends on the following items: None found

Items that depend on this one: None found


Carbon-Copy List
  • -email is unavailable- added by godfrey (Posted a comment)
  • -email is unavailable- added by rik5 (Posted a comment)
  • -email is unavailable- added by sebald (Submitted the item)

    Do you think this task is very important?
    If so, you can add your encouragement to it.
    This task has 0 encouragements so far.

    Only project members can vote.




    Follow 3 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2018-04-16 rik5 StatusNone => Fixed
        Open/ClosedOpen => Closed
    2018-04-15 sebald Attached File- => Added octave-opengl_qt_toolkit_line_stipple_nouveau_driver-djs2018apr15.patch, #43931

    Back to the top

    Powered by Savane 3.4