bugGNU Octave - Bugs: bug #47914, segfault with OpenGL patches and...

 
 

bug #47914: segfault with OpenGL patches and address sanitizer

Submitted by:  Rik <rik5>
Submitted on:  Fri 13 May 2016 05:43:26 PM UTC  
 
Category: Plotting with OpenGLSeverity: 4 - Important
Priority: 5 - NormalItem Group: Segfault, Bus Error, etc.
Status: Works For MeAssigned to: None
Originator Name: Open/Closed: Closed
Release: devOperating System: GNU/Linux

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

(Jump to the original submission Jump to the original submission)

Mon 04 Sep 2017 04:53:54 AM UTC, comment #20:

I just did 'rundemos' on scripts/plot/(draw|appearance/utils) and using ASAN options

and there were no memory leaks or segfaults. I'm going to close this bug and a new one can be opened new leaks are found.

Rik <rik5>
Project Administrator
Sun 03 Sep 2017 03:46:01 PM UTC, comment #19:

I'm pretty certain now this is a bug in the SW renderer. I ran the development version recently and the stack trace showed that the problem was with disabling clip planes.

I used printf debug statements to find that it was the very first call to

which was causing the heap-buffer-overflow.

I get this only when I have set LIBGL_ALWAYS_SOFTWARE to 1. If I unset this environment variable then the ASAN error goes away.

Rik <rik5>
Project Administrator
Mon 29 Aug 2016 10:18:22 PM UTC, comment #18:

Hi,

I'm a random octave user, and I am finding this in my own C++ applications when compiled with ubsan. I have the same backtrace (different symbol position, but same ubsan error and stack depth).

I found this bug when searching for the crash message - perhaps this is indeed a problem with the video driver...

I'm using debian testing (x86_64), octave 4.0.3-1, as a virtualbox guest (4.3.30 r101610).

I've fired off a rebuild of the octave package with ubsan enabled, but it looks like it might take a while - I'll get back when I have a test package.

$ glxinfo |head -n 60
libGL error: pci id for fd 4: 80ee:beef, driver (null)
OpenGL Warning: Failed to connect to host. Make sure 3D acceleration is enabled for this VM.
libGL error: core dri or dri2 extension not found
libGL error: failed to load driver: vboxvideo
name of display: :0.0
display: :0 screen: 0
direct rendering: Yes
server glx vendor string: SGI
server glx version string: 1.4
server glx extensions:
GLX_ARB_multisample, GLX_EXT_import_context, GLX_EXT_texture_from_pixmap,
GLX_EXT_visual_info, GLX_EXT_visual_rating, GLX_MESA_copy_sub_buffer,
GLX_OML_swap_method, GLX_SGIS_multisample, GLX_SGIX_fbconfig,
GLX_SGIX_pbuffer, GLX_SGI_make_current_read
client glx vendor string: Mesa Project and SGI
client glx version string: 1.4
client glx extensions:
GLX_ARB_create_context, GLX_ARB_create_context_profile,
GLX_ARB_create_context_robustness, GLX_ARB_fbconfig_float,
GLX_ARB_framebuffer_sRGB, GLX_ARB_get_proc_address, GLX_ARB_multisample,
GLX_EXT_buffer_age, GLX_EXT_create_context_es2_profile,
GLX_EXT_create_context_es_profile, GLX_EXT_fbconfig_packed_float,
GLX_EXT_framebuffer_sRGB, GLX_EXT_import_context,
GLX_EXT_texture_from_pixmap, GLX_EXT_visual_info, GLX_EXT_visual_rating,
GLX_INTEL_swap_event, GLX_MESA_copy_sub_buffer,
GLX_MESA_multithread_makecurrent, GLX_MESA_query_renderer,
GLX_MESA_swap_control, GLX_OML_swap_method, GLX_OML_sync_control,
GLX_SGIS_multisample, GLX_SGIX_fbconfig, GLX_SGIX_pbuffer,
GLX_SGIX_visual_select_group, GLX_SGI_make_current_read,
GLX_SGI_swap_control, GLX_SGI_video_sync
GLX version: 1.4
GLX extensions:
GLX_ARB_get_proc_address, GLX_ARB_multisample, GLX_EXT_import_context,
GLX_EXT_texture_from_pixmap, GLX_EXT_visual_info, GLX_EXT_visual_rating,
GLX_MESA_copy_sub_buffer, GLX_MESA_multithread_makecurrent,
GLX_MESA_query_renderer, GLX_OML_swap_method, GLX_SGIS_multisample,
GLX_SGIX_fbconfig, GLX_SGIX_pbuffer, GLX_SGI_make_current_read
Extended renderer info (GLX_MESA_query_renderer):
Vendor: VMware, Inc. (0xffffffff)
Device: llvmpipe (LLVM 3.8, 128 bits) (0xffffffff)
Version: 11.2.2
Accelerated: no
Video memory: 2971MB
Unified memory: no
Preferred profile: core (0x1)
Max core profile version: 3.3
Max compat profile version: 3.0
Max GLES1 profile version: 1.1
Max GLES[23] profile version: 3.0
OpenGL vendor string: VMware, Inc.
OpenGL renderer string: Gallium 0.4 on llvmpipe (LLVM 3.8, 128 bits)
OpenGL version string: 3.0 Mesa 11.2.2
OpenGL shading language version string: 1.30
OpenGL context flags: (none)
OpenGL extensions:
GL_AMD_conservative_depth, GL_AMD_draw_buffers_blend,
GL_AMD_seamless_cubemap_per_texture, GL_AMD_shader_stencil_export,
GL_AMD_shader_trinary_minmax, GL_ANGLE_texture_compression_dxt3,
GL_ANGLE_texture_compression_dxt5, GL_APPLE_packed_pixels,
GL_APPLE_vertex_array_object, GL_ARB_ES2_compatibility,
GL_ARB_ES3_compatibility, GL_ARB_arrays_of_arrays, GL_ARB_base_instance,
GL_ARB_blend_func_extended, GL_ARB_buffer_storage,
GL_ARB_clear_buffer_object, GL_ARB_clip_control,
GL_ARB_color_buffer_float, GL_ARB_compressed_texture_pixel_storage,

Person <mycae>
Fri 29 Jul 2016 10:09:14 PM UTC, comment #17:

Still failing with the patch. Here is the traceback I see from the address sanitizer:

John W. Eaton <jwe>
Project Administrator
Thu 28 Jul 2016 04:45:55 PM UTC, comment #16:

That error might be caused by the current implementation of lights. On one of my systems, the constant GL_MAX_LIGHTS is way too high. I am not sure what happens in the following lines when "i" increases way too far:

A fix for this is contained in file #37999 in patch #8943.

It might also be completely unrelated but comment #14 rang a bell to me.

Markus Mützel <mmuetzel>
Project Member
Thu 28 Jul 2016 12:53:32 PM UTC, comment #15:

BTW, I also see that this error happens if I set graphics_toolkit to fltk, so it seems more like a bug in OpenGL an error in the way we are calling OpenGL routines rather than memory corruption caused by threading in the Qt graphics code.

John W. Eaton <jwe>
Project Administrator
Thu 28 Jul 2016 06:27:40 AM UTC, comment #14:

Oh, I see. That also causes a crash for me. The traceback I get says that it is happening in the function call

in opengl_renderer::draw_axes_children.

John W. Eaton <jwe>
Project Administrator
Thu 28 Jul 2016 06:14:04 AM UTC, comment #13:

I can reproduce this problem if i set LIBGL_ALWAYS_SOFTWARE=1:

ASAN_OPTIONS=alloc_dealloc_mismatch=0 LIBGL_ALWAYS_SOFTWARE=1 ./run-octave --no-gui

octave:1> demo patch 2
patch example 2:
%% Unclosed patch
clf;
t1 = (1/16:1/8:1)' * 2*pi;
t2 = ((1/16:1/16:1)' + 1/32) * 2*pi;
x1 = sin (t1) - 0.8;
y1 = cos (t1);
x2 = sin (t2) + 0.8;
y2 = cos (t2);
patch ([[x1;NaN(8,1)],x2], [[y1;NaN(8,1)],y2], 'r');

octave:2> =================================================================
==22330==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x6160004418c0 at pc 0x7fd08ca29ccf bp 0x7ffe8b3cd320 sp 0x7ffe8b3ccac8
READ of size 52 at 0x6160004418c0 thread T0
#0 0x7fd08ca29cce (/lib64/libasan.so.3+0x5ecce)
#1 0x7fd0576d9d23 (/usr/lib64/dri/swrast_dri.so+0x36bd23)
....
SUMMARY: AddressSanitizer: heap-buffer-overflow (/lib64/libasan.so.3+0x5ecce)
Shadow bytes around the buggy address:
0x0c2c800802c0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c2c800802d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0c2c800802e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0c2c800802f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0c2c80080300: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x0c2c80080310: 00 00 00 00 00 00 00 00[fa]fa fa fa fa fa fa fa
0x0c2c80080320: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c2c80080330: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c2c80080340: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c2c80080350: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c2c80080360: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Heap right redzone: fb
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack partial redzone: f4
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
==22330==ABORTING

Dmitri A. Sergatskov <dasergatskov>
Thu 28 Jul 2016 04:10:41 AM UTC, comment #12:

I can't duplicate this problem either. I compiled with the address sanitizer options and default compiler flags and also "-O0 -ggdb3" and I did not see a crash. Memory leaks were reported when I exited Octave, but that's it.

Also, I still can't start the GUI when compiling with the sanitizer flags.

Maybe this is just buggy OpenGL drivers or libraries? I don't know what we can do about that. I have the following:

John W. Eaton <jwe>
Project Administrator
Tue 12 Jul 2016 08:08:18 PM UTC, comment #11:

changeset: 22099:4c0f78b3c86f

I cannot complete the doc build if I compile with -fsanitize=address ; there are too many leaks building voronoi.txt.
It gives up with
SUMMARY: AddressSanitizer: 126332 byte(s) leaked in 2675 allocation(s).
Makefile:27702: recipe for target 'doc/interpreter/voronoi.txt' failed

(I had to set ASAN_OPTIONS=new_delete_type_mismatch=0 or I get
SUMMARY: AddressSanitizer: new-delete-type-mismatch (/lib64/libasan.so.3+0xc8b60) in operator delete(void*, unsigned long)
)

I could run
ASAN_OPTIONS=new_delete_type_mismatch=0 ./run-octave --no-gui
and execute 'demo patch 2" w/ no problem after that)

This is on ccomputer with nvidia card:

glxinfo -B
name of display: :0
display: :0 screen: 0
direct rendering: Yes
Extended renderer info (GLX_MESA_query_renderer):
Vendor: nouveau (0x10de)
Device: NVC3 (0xdd8)
Version: 11.2.2
Accelerated: yes
Video memory: 995MB
Unified memory: no
Preferred profile: core (0x1)
Max core profile version: 4.1
Max compat profile version: 3.0
Max GLES1 profile version: 1.1
Max GLES[23] profile version: 3.0
OpenGL vendor string: nouveau
OpenGL renderer string: Gallium 0.4 on NVC3
OpenGL core profile version string: 4.1 (Core Profile) Mesa 11.2.2 (git-5de088f)
OpenGL core profile shading language version string: 4.10
OpenGL core profile context flags: (none)
OpenGL core profile profile mask: core profile

OpenGL version string: 3.0 Mesa 11.2.2 (git-5de088f)
OpenGL shading language version string: 1.30
OpenGL context flags: (none)

OpenGL ES profile version string: OpenGL ES 3.0 Mesa 11.2.2 (git-5de088f)
OpenGL ES profile shading language version string: OpenGL ES GLSL ES 3.00

Please tell me if I am doing something wrong. I will try on different hardware later today.

Dmitri.
--

Dmitri A. Sergatskov <dasergatskov>
Tue 12 Jul 2016 05:39:05 PM UTC, comment #10:

Very possible.

Anyways, I tested with regular -O2 options and it still crashes.

Rik <rik5>
Project Administrator
Tue 12 Jul 2016 05:24:57 PM UTC, comment #9:

It might also be the mesa driver for your particular system, or the version of mesa, since I see swrast_dri.so at the top of the stack where the buffer overflow is reported.

Mike Miller <mtmiller>
Project Administrator
Tue 12 Jul 2016 05:19:03 PM UTC, comment #8:

It might be my version of gcc (Ubuntu 5.2.1-22ubuntu2) among other things.

Rik <rik5>
Project Administrator
Tue 12 Jul 2016 05:09:57 PM UTC, comment #7:

This still fails for me with cset 5ad67277b007. This is a debug version so I am setting '-O0 -g' in the flags. I'll try again with a regular build to see if optimizations stop the segfault.

Rik <rik5>
Project Administrator
Tue 12 Jul 2016 05:02:59 PM UTC, comment #6:

I build with the following configure options for gcc's address sanitizer:

I get many leaks detected, but no heap-buffer-overflow.

Mike Miller <mtmiller>
Project Administrator
Tue 12 Jul 2016 01:10:59 PM UTC, comment #5:

I checked that "-fsanitize=address" was properly added in the config.log but still, after rebuilding, I couldn't trigger the heap-buffer-overflow by simply running "demo patch 2". Anything more I should do?

Pantxo Diribarne <pantxo>
Project Member
Tue 12 Jul 2016 12:18:30 PM UTC, comment #4:

Sorry, spoke too soon ... I copied the configure option from comment #1 and ran "make" without checking the config log. Actually it should be "--enable-address-sanitizer-flags". Rebuilding now.

Pantxo Diribarne <pantxo>
Project Member
Tue 12 Jul 2016 11:49:28 AM UTC, comment #3:

@Rik: I tried to reproduce this but I am not sure about the way to go. I configured octave with "../octave/configure --enable-address-sanitizer" and then rebuilt the default branch. Simply running the demo after "./run-octave -f --no-gui" is not enough to trigger a bug. Is there any flag we must pass to run-octave in order to enable the address checking? Should I run octave from gdb?

Pantxo Diribarne <pantxo>
Project Member
Fri 13 May 2016 06:22:15 PM UTC, comment #2:

I'm guessing that they are related. Why don't you file a separate bug report about that, and then we can link it as a dependency of this report.

Rik <rik5>
Project Administrator
Fri 13 May 2016 06:14:19 PM UTC, comment #1:

Talking about sanitizing and patch:
patch() doesn't check input sufficiently rigorously.
In case of "patch ('vertices', f, 'faces', ...)" calls, when f contains NaN rows, Octave segfaults (very) hard. Should I file a separate bug report for that?

Philip Nienhuis <philipnienhuis>
Project Member
Fri 13 May 2016 05:43:26 PM UTC, original submission:

When Octave has been configured with the --enable-address-sanitizer option, the second patch demo causes a segmentation violation. The log is shown below.

Rik <rik5>
Project Administrator

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by mycae (Posted a comment)
  • -unavailable- added by mmuetzel (Posted a comment)
  • -unavailable- added by jwe (Posted a comment)
  • -unavailable- added by dasergatskov (Posted a comment)
  • -unavailable- added by mtmiller (Posted a comment)
  • -unavailable- added by pantxo (Posted a comment)
  • -unavailable- added by rik5 (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only project members can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 4 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Mon 04 Sep 2017 04:53:54 AM UTCrik5StatusConfirmed=>Works For Me
      Open/ClosedOpen=>Closed
    Fri 01 Jul 2016 05:26:23 PM UTCrik5Severity3 - Normal=>4 - Important
    Sun 15 May 2016 02:44:33 PM UTCrik5CategoryInterpreter=>Plotting with OpenGL

    Back to the top


    Powered by Savane 3.1-cleanup1