bugGNU Octave - Bugs: bug #51631, make check fails at __opengl_info__

 
 

bug #51631: make check fails at __opengl_info__

Submitter:  Dmitri A. Sergatskov <dasergatskov>
Submitted:  Mon 31 Jul 2017 06:13:04 PM UTC
   
 
Category:  Test Suite Severity:  3 - Normal
Priority:  5 - Normal Item Group:  None
Status:  Fixed Assigned to:  None
Originator Name:  Open/Closed:  * Closed
Release:  * dev Operating System:  * GNU/Linux
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Fri 18 Aug 2017 12:02:18 AM UTC, comment #31: 

What i also noticed that running e.g. glxinfo -B through xvfb takes a long time

time xvfb-run -a -s '-screen 0 640x480x24' glxinfo -B
name of display: :99
display: :99  screen: 0
direct rendering: Yes
Extended renderer info (GLX_MESA_query_renderer):
    Vendor: VMware, Inc. (0xffffffff)
    Device: llvmpipe (LLVM 4.0, 256 bits) (0xffffffff)
    Version: 17.1.5
    Accelerated: no
    Video memory: 15925MB
    Unified memory: no
    Preferred profile: compat (0x2)
    Max core profile version: 0.0
    Max compat profile version: 2.1
    Max GLES1 profile version: 1.1
    Max GLES[23] profile version: 2.0
OpenGL vendor string: VMware, Inc.
OpenGL renderer string: Gallium 0.4 on llvmpipe (LLVM 4.0, 256 bits)
OpenGL version string: 2.1 Mesa 17.1.5
OpenGL shading language version string: 1.30

OpenGL ES profile version string: OpenGL ES 2.0 Mesa 17.1.5
OpenGL ES profile shading language version string: OpenGL ES GLSL ES 1.0.16


real        0m3.043s
user        0m0.020s
sys        0m0.023s

=========

time glxinfo -B
name of display: :0
display: :0  screen: 0
direct rendering: Yes
Extended renderer info (GLX_MESA_query_renderer):
    Vendor: X.Org (0x1002)
    Device: AMD OLAND (DRM 2.50.0 / 4.12.5-300.fc26.x86_64, LLVM 4.0.0) (0x6610)
    Version: 17.1.5
    Accelerated: yes
    Video memory: 2048MB
    Unified memory: no
    Preferred profile: core (0x1)
    Max core profile version: 4.5
    Max compat profile version: 3.0
    Max GLES1 profile version: 1.1
    Max GLES[23] profile version: 3.1
OpenGL vendor string: X.Org
OpenGL renderer string: Gallium 0.4 on AMD OLAND (DRM 2.50.0 / 4.12.5-300.fc26.x86_64, LLVM 4.0.0)
OpenGL core profile version string: 4.5 (Core Profile) Mesa 17.1.5
OpenGL core profile shading language version string: 4.50
OpenGL core profile context flags: (none)
OpenGL core profile profile mask: core profile

OpenGL version string: 3.0 Mesa 17.1.5
OpenGL shading language version string: 1.30
OpenGL context flags: (none)

OpenGL ES profile version string: OpenGL ES 3.1 Mesa 17.1.5
OpenGL ES profile shading language version string: OpenGL ES GLSL ES 3.10


real        0m0.030s
user        0m0.018s
sys        0m0.009s


may be that somehow triggers the bug.

Dmitri.

Dmitri A. Sergatskov <dasergatskov>
Thu 17 Aug 2017 11:57:47 PM UTC, comment #30: 

I think there is some race condition bug(s) still remains.
When i do dump_plot_demos (or similar) I get sporadic (non-reproducible) problems. Usually it is just white windows (no plots shown) yet a png file is generated. The other day computer hanged with 100%CPU load in octave process. Killing the process
and restarting the script allows it to complete.
But buildbot seems to run fine.

Dmitri.
--

Dmitri A. Sergatskov <dasergatskov>
Thu 17 Aug 2017 11:36:12 PM UTC, comment #29: 

I am still getting very frequent intermittent test failures like this on the Debian buildbot which runs 'make check' under Xvfb. Around 10 out of the last 50 builds, over about a period of a week, all after this change, failed on this test:

http://buildbot.octave.org:8010/builders/gcc-6-debian?numbuilds=50

Is there anything we can do to make the build system and test suite more reliable on a headless build system?

Mike Miller <mtmiller>
Group Member
Mon 07 Aug 2017 04:24:52 PM UTC, comment #28: 

Looks good enough to me.

Dmitri.
--

Dmitri A. Sergatskov <dasergatskov>
Mon 07 Aug 2017 04:11:48 PM UTC, comment #27: 

Is the current hack good enough to close this bug report?

It seems like another bug report, or even a project page, should be opened on race conditions in the graphics code.

Rik <rik5>
Group administrator
Mon 07 Aug 2017 03:58:03 AM UTC, comment #26: 

I think it is the same problem as bug #50479
https://savannah.gnu.org/bugs/?50479
(See comment #34).

Dmitri.
--

Dmitri A. Sergatskov <dasergatskov>
Mon 07 Aug 2017 03:53:19 AM UTC, comment #25: 

FYI On ubuntu 16.04 I can run make check without any problems.
hg id 9f31de57ad36

Doug Stewart <dastew>
Mon 07 Aug 2017 01:34:31 AM UTC, comment #24: 

I have a compile w/o qt (fltk only) and it is the same.
I assume this is the issue with Fedora doing something funny with
mesa.

Dmitri.
--

Dmitri A. Sergatskov <dasergatskov>
Mon 07 Aug 2017 01:25:09 AM UTC, comment #23: 

My guess is that this is a problem with the Qt toolkit.  Can you edit _opengl_info_.m and comment out the Qt test?  Just add a single '#' to the first line of the test like so


%!#testif HAVE_OPENGL, HAVE_QT; have_window_system () && any (strcmp ("qt", available_graphics_toolkits ()))


If it works then that will give an indication of where to start searching for the problem.

Rik <rik5>
Group administrator
Sun 06 Aug 2017 05:23:44 PM UTC, comment #22: 

"make check" hangs, "LD_PRELOAD=/usr/lib64/libGLX_mesa.so.0 make check" passes.

Also, "test _opengl_info_" at octave prompt passes in all cases.

Dmitri.
--

Dmitri A. Sergatskov <dasergatskov>
Sun 06 Aug 2017 04:29:02 PM UTC, comment #21: 

Now my computer hangs with 100% CPU load at make check
(in the scripts/plot/util/__opengl_info__.m).

I am re-cloning and rebuilding just to make sure...

Dmitri.
--

Dmitri A. Sergatskov <dasergatskov>
Sun 06 Aug 2017 02:49:26 PM UTC, comment #20: 

I checked in a changeset that adds the refresh statement (http://hg.savannah.gnu.org/hgweb/octave/rev/ca4cca5710dc).  This seems to help significantly with reducing the failures during 'make test'.  I also re-wrote the BIST tests so that they are executed once with the FLTK toolkit and once with the Qt toolkit if they are available.  The previous code was kind of silly in that it ran the same test twice using the same toolkit.

Rik <rik5>
Group administrator
Wed 02 Aug 2017 02:53:50 PM UTC, comment #19: 

Adding a pause after the refresh seems to help for me but I don't really understand yet why.

John W. Eaton <jwe>
Group administrator
Wed 02 Aug 2017 01:01:14 AM UTC, comment #18: 

Ick!  There are definitely weird timing patterns. 

The graphic properties are maintained in graphics.cc and graphics-props.cc and they exist independently of the backend.  This allows Octave to support a toolkit like gnuplot, but it also means there can be consistency issues.  In this case, the backend toolkit itself is calculating values, like _gl_version_, and then assigning to the property database.  If the Octave interpreter queries the property database before the toolkit has updated the values then it gets bad results.

I added some debug statements to Fdrawnow to see when the refresh() call was actually forcing the graphics system to redraw things.  I also added a debug statement in graphics-props.cc to check when _gl_version_ was being queried with a get statement.

The m-file code was


octave:12> hf = figure; hax = axes; pause (2); get (hf, "__gl_version__")
drawnow_executing: 0
getting __gl_version
ans =
octave:13> draw_figure


As you can see, the get() statement is executing before the call to draw_figure and a large pause makes no difference at all.  It seems that pause halt the entire execution thread, including the FLTK toolkit.  A refresh() statement by itself is not enough.


octave:21>     hf = figure; hax = axes; refresh (hf); get (hf, "__gl_version__")
drawnow_executing: 0
getting __gl_version
ans =
octave:22> draw_figure


But reset with a pause is


octave:23>     hf = figure; hax = axes; pause (.1); refresh (hf); get (hf, "__gl_version__")
drawnow_executing: 0
drawnow_executing: 0
draw_figure
getting __gl_version
ans = 3.0 Mesa 17.0.7




Rik <rik5>
Group administrator
Wed 02 Aug 2017 12:32:52 AM UTC, comment #17: 

@jwe: It is definitely a race condition.  I had experimented a bit with removing or reducing the pause() before the refresh in _opengl_info_.m and I got worse results (more failures) as the delay was reduced.


    hf = figure ("position", [0,0,1,1], "toolbar", "none", "menubar", "none");
    hax = axes ();
    pause (0.2);  # <-- altered this delay
+    refresh (hf);


The delay didn't have to be large, .01 was enough most of the time.  But it will vary from machine to machine, etc.

Could the refresh routine (drawnow) be changed so that it doesn't return from C++ until the updates have taken place?



Rik <rik5>
Group administrator
Wed 02 Aug 2017 12:02:39 AM UTC, comment #16: 

Rik: OK, I see now that _opengl_info_ is taking care of creating a figure so we can get the OpenGL info.  If refresh is needed for fltk, then maybe the simplest thing is just to add that just after the "pause (0.2)" line in the gl_info function in _opengl_info_.m.  Does that work for you?  It allowed me to run


./run-octave --eval --norc "graphics_toolkit fltk; __opengl_info__"
./run-octave --no-gui --norc --eval "graphics_toolkit fltk; __opengl_info__"
./run-octave --no-gui-libs --norc --eval "graphics_toolkit fltk; __opengl_info__"
./run-octave -cli --norc --eval "graphics_toolkit fltk; __opengl_info__"


However, after repeated tests, it sometimes (but not always) fails when using --no-gui-libs.  That, once again, seems like some kind of timing issue?


John W. Eaton <jwe>
Group administrator
Tue 01 Aug 2017 11:28:26 PM UTC, comment #15: 

Verified that 'make check' now passes, but there is still an issue with FLTK.  For example,


./run-octave -f --no-gui-libs   # CLI-version only has FLTK available
test __opengl_info__


will fail.  But, this is really related to having an OpenGL window available.  This will pass


./run-octave -f --no-gui-libs
plot (1:10);
test __opengl_info__


The problem seems to be in gl-render.cc where the special variables such as __gl_version are set.  The code is


  void
  opengl_renderer::draw_figure (const figure::properties& props)
  {
    // Initialize OpenGL context

    init_gl_context (props.is_graphicssmoothing (), props.get_color_rgb ());

#if defined (HAVE_OPENGL)
    std::cerr << "draw_figure" <<std::endl;
    props.set___gl_extensions__ (gl_get_string (GL_EXTENSIONS));
    props.set___gl_renderer__ (gl_get_string (GL_RENDERER));
    props.set___gl_vendor__ (gl_get_string (GL_VENDOR));
    props.set___gl_version__ (gl_get_string (GL_VERSION));

#endif

    // Draw children

    draw (props.get_all_children (), false);
  }


I modified it to add the output on stderr whenever the routine is called.  This will cause the function to be executed.


close all
graphics_toolkit qt
h = figure ();


However, this will not.


close all
graphics_toolkit fltk
h = figure ();


A cheap hack for _opengl_info_.m is to do some operation which forces a redraw of the figure.  Apparently drawnow(), pause() are not enough.  But a refresh command is.  This reliably works for me.


    hf = figure ("position", [0,0,1,1], "toolbar", "none", "menubar", "none");
    hax = axes ();
    pause (0.2);
+    refresh (hf);




Rik <rik5>
Group administrator
Tue 01 Aug 2017 11:16:53 PM UTC, comment #14: 

I'd preter to understand why it is failing for fltk and fix that instead.

John W. Eaton <jwe>
Group administrator
Tue 01 Aug 2017 09:05:50 PM UTC, comment #13: 

This is with fltk, but no Qt


octave:1> test __opengl_info__
***** testif HAVE_OPENGL, HAVE_QT; have_window_system
 a = __opengl_info__ ();
 assert (! isempty (a))
 assert (isfield (a, "version"))
----- skipped test (missing feature)

PASSES 2 out of 2 tests
Skipped 1 test due to missing features
octave:2> test __opengl_info__
warning: __opengl_info__: can not obtain OpenGL information
warning: __opengl_info__: some elements in list of return values are undefined
***** testif HAVE_OPENGL, HAVE_FLTK; have_window_system
 a = __opengl_info__ ();
 assert (! isempty (a))
 assert (isfield (a, "version"))
!!!!! test failed
assert (!isempty (a)) failed


glxinfo works. Could we pipe its output to a file and
extract the necessary info from there?

Dmitri.
--

Dmitri A. Sergatskov <dasergatskov>
Tue 01 Aug 2017 07:49:58 PM UTC, comment #12: 

Dmitri: Probably, though I haven't tried it.

John W. Eaton <jwe>
Group administrator
Tue 01 Aug 2017 07:35:30 PM UTC, comment #11: 

I am still compiling, but
WRT comment #9, does it mean the test will fail if only fltk
compiled in and qt is not?

Dmitri.
--

Dmitri A. Sergatskov <dasergatskov>
Tue 01 Aug 2017 07:32:20 PM UTC, comment #10: 

that fixed it for me.

Doug Stewart <dastew>
Tue 01 Aug 2017 06:49:40 PM UTC, comment #9: 

I checked in the following changeset:

http://hg.savannah.gnu.org/hgweb/octave/rev/5d8ef9b859f8

Does that fix the problem for you?

I see that the following works now with the qt toolkit, but not with fltk:


./run-octave --norc --eval "graphics_toolkit qt; __opengl_info__"
./run-octave --norc --eval "graphics_toolkit fltk; __opengl_info__"


I think this is because the OpenGL context is persistent with the qt toolkit, but it is initiailized and destroyed with each figure when using the fltk toolkit.  So in that case, you need to have a figure open to get anything back from _opengl_info_.


John W. Eaton <jwe>
Group administrator
Mon 31 Jul 2017 10:31:24 PM UTC, comment #8: 

Thanks, given that I think I can fix it pretty quickly.

John W. Eaton <jwe>
Group administrator
Mon 31 Jul 2017 09:42:58 PM UTC, comment #7: 

I bisected and found that this is the first one that fails


Changeset:
23773 (5ff24576b783) defer installing qt graphics toolkit until interpreter is ready …


Doug Stewart <dastew>
Mon 31 Jul 2017 08:54:16 PM UTC, comment #6: 

Just chiming in, but this has been failing for me for a few weeks now.

When using the GUI, this always works


./run-octave -f
test __opengl_info__
test __opengl_info__
test __opengl_info__


This fails, sometimes on the first go, but always on the second and third.


./run-octave -f --no-gui-libs
test __opengl_info__
test __opengl_info__
test __opengl_info__



Rik <rik5>
Group administrator
Mon 31 Jul 2017 08:22:40 PM UTC, comment #5: 

If I remove ";" at the end of a= _opengl_info_() line I see this:


scripts/plot/util/__opengl_info__.m .........................a =
  scalar structure containing the fields:
    version = 3.0 Mesa 17.1.5
    vendor = nouveau
    renderer = Gallium 0.4 on NVC3
    extensions =
    {
      [1,1] = GL_ARB_multisample
      [1,2] = GL_EXT_abgr
  .....deleted..........
      [1,233] = GL_MESA_shader_integer_functions
    }

a = [](0x0)
 PASS      2/3
                                                                  FAIL    1


Dmitri.

Dmitri A. Sergatskov <dasergatskov>
Mon 31 Jul 2017 07:33:52 PM UTC, comment #4: 

Sorry -- pause(1.0) was added by me (it did not change anything).
Note that it is always "HAVE_QT" fails after it passes
"HAVE_FLTK" test.

Dmitri.

Dmitri A. Sergatskov <dasergatskov>
Mon 31 Jul 2017 06:34:56 PM UTC, comment #3: 

It always fails for me.

Doug Stewart <dastew>
Mon 31 Jul 2017 06:33:35 PM UTC, comment #2: 

Does it always fail, or only sometimes?

I've noticed occasional failures like this with the buildbot builds, which use xvfb to provide the X11 display.

Maybe OpenGL is not completely initialized when the call requesting info is made?  I'm not sure what the purpose of the pause is either.  I don't see what difference it could make since it occurs after the call to _opengl_info_.

John W. Eaton <jwe>
Group administrator
Mon 31 Jul 2017 06:16:57 PM UTC, comment #1: 

I see the same thing here on ubuntu 16.04 and development branch

Doug Stewart <dastew>
Mon 31 Jul 2017 06:13:04 PM UTC, original submission:  

hg id
3ac5d3d01cad+ tip @
 
make check fails at

 scripts/plot/util/__opengl_info__.m ......................... PASS      2/3
                                                                  FAIL    1


Also in test/fntests.log I see


>>>>> processing /d2/home/dima/src/octave/scripts/plot/util/__opengl_info__.m
***** testif HAVE_OPENGL, HAVE_QT; have_window_system
 a = __opengl_info__ ();
 pause(1.0)
 assert (! isempty (a))
 assert (isfield (a, "version"))
!!!!! test failed
assert (!isempty (a)) failed


yet if is start ./run-octave (with or without --no-gui)


octave:1> test __opengl_info__ verbose
>>>>> /d2/home/dima/src/octave/gcc_def/../scripts/plot/util/__opengl_info__.m
***** testif HAVE_OPENGL, HAVE_FLTK; have_window_system
 a = __opengl_info__ ();
 assert (! isempty (a))
 assert (isfield (a, "version"))
***** testif HAVE_OPENGL, HAVE_QT; have_window_system
 a = __opengl_info__ ();
 pause(1.0)
 assert (! isempty (a))
 assert (isfield (a, "version"))
***** error __opengl_info ("foobar")
PASSES 3 out of 3 tests


This is both on Fedora 26 (mesa-17.1.5) and RHEL-7.4
(mesa-11.2.2)

Dmitri.
 

Dmitri A. Sergatskov <dasergatskov>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by mtmiller (Posted a comment)
  • -email is unavailable- added by rik5 (Posted a comment)
  • -email is unavailable- added by jwe (Posted a comment)
  • -email is unavailable- added by dastew (Posted a comment)
  • -email is unavailable- added by dasergatskov (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 4 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2017-08-07 rik5 StatusReady For Test Fixed
        Open/ClosedOpen Closed
    2017-08-01 jwe StatusConfirmed Ready For Test
    2017-07-31 rik5 StatusNone Confirmed

    Back to the top

    Powered by Savane 3.13-f8d8.
    Corresponding source code