bugGNU Octave - Bugs: bug #44916, --softwareopengl option for...

 
 

bug #44916: --softwareopengl option for performance

Submitter:  Martin Kunz <mkunz>
Submitted:  Fri 24 Apr 2015 10:51:58 AM UTC
   
 
Category:  Plotting with OpenGL Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Performance
Status:  None Assigned to:  None
Originator Name:  Open/Closed:  * Open
Release:  * dev Operating System:  * Microsoft Windows
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Tue 28 Aug 2018 04:55:09 PM UTC, comment #82: 

Sure, I think you can do that, provided that you are referencing all the symbols through pointers.  So when you load the new library, you reassign all the pointers so that you call functions from the new library.  I assume that's what Qt's QOpenGLFunction class does.  It's also the way the JVM is supposed to work, but good luck unloading/reloading that one!  Well, it didn't seem to work for me anyway...

John W. Eaton <jwe>
Group administrator
Tue 28 Aug 2018 04:04:40 PM UTC, comment #81: 

I wonder how the Mathworks implemented switching between hardware and software rendering on the run (on Windows):
https://de.mathworks.com/help/matlab/ref/opengl.html

Can we unload one dynamic library and load a different one that provides the same symbols?

Markus Mützel <mmuetzel>
Group administrator
Tue 28 Aug 2018 01:08:01 PM UTC, comment #80: 

For Windows systems, building Qt with the configure option "-opengl dynamic" and installing our version of the OpenGL library as "opengl32sw.dll" may be of some help, but will apparently also require that all accesses of OpenGL calls from Octave be done through Qt's QOpenGLFunctions class.  Changing the build system is easy.  Changing the way we call OpenGL functions will be somewhat more difficult.

I don't know whether there is a similar solution for non-Windows systems.

John W. Eaton <jwe>
Group administrator
Tue 09 Aug 2016 07:04:00 AM UTC, comment #79: 

Rik: Just a minor point about Matlab behavior. Matlab
switches to software OpenGL if it finds that OpenGL
is not available, such as for remote plotting. Someday,
similar behavior for Octave may be a good choice in addition
to the user option. The current choice is right for 4.2
at least.

Michael Godfrey <godfrey>
Group Member
Mon 08 Aug 2016 07:25:04 PM UTC, comment #78: 

I have reduced the severity of this bug to 3--It is no longer a blocker for the 4.2 release.  The outstanding issue is that all OpenGL Qt rendering has been switched over to use software rather than hardware.  The switchover should be configurable so that users with HW that works can get the performance benefit of it.  At least in Matlab, HW rendering is used by default unless the --softwareopengl command line switch is used.

Rik <rik5>
Group administrator
Mon 08 Aug 2016 07:19:59 PM UTC, comment #77: 

The minor problem reported by Michael Godfrey seems to be an issue whenever the graphics toolkit is not qt.  Either gnuplot, or fltk, will produce the warning.


./run-octave -f --no-gui
graphics_toolkit fltk
__opengl_info__
warning: __opengl_info__: can not obtain OpenGL information
warning: called from
    __opengl_info__ at line 64 column 5


It seems that FLTK needs to have an object actually drawn before the OpenGL fields get populated in properties.  I pushed a cset here (http://hg.savannah.gnu.org/hgweb/octave/rev/fddc5604d1fa) which draws an axes object and then calls drawnow(), rather than waitfor with a 1 second delay, in order to force the graphics system to populate the fields.

Rik <rik5>
Group administrator
Sat 06 Aug 2016 12:57:11 PM UTC, comment #76: 

The gnuplot toolkit doesn't use opengl (at  in a way that octave has access to) so will print that.

John Donoghue <lostbard>
Group Member
Sat 06 Aug 2016 01:16:24 AM UTC, comment #75: 

@Michael, fyi:

octave:1> graphics_toolkit gnuplot
octave:2> _opengl_info_
warning: _opengl_info_: can not obtain OpenGL information
warning: called from
    _opengl_info_ at line 64 column 5

Dmitri.

Dmitri A. Sergatskov <dasergatskov>
Fri 05 Aug 2016 11:23:22 PM UTC, comment #74: 

John,
It is something in one of my init files. If I run
octave --no-gui --no-init-file --norc
then it works correctly. And,

octave --no-gui --no-init-file
also works. What does --no-init-file turn off?

I will isolate what it is soon. But, not tonight.

Michael Godfrey <godfrey>
Group Member
Fri 05 Aug 2016 08:42:32 PM UTC, comment #73: 

Michael:  That's strange, it works for me.  The following lines in the function attempt to create a figure if none already exists:


  if (isempty (info))
    h = figure ("position", [0,0,1,1], "toolbar", "none", "menubar", "none");
    waitfor (h, "timeout", 1);
    info = fig_gl_info (h);
    close (h);
  endif


I'm not sure what is preventing this from working for you.

John W. Eaton <jwe>
Group administrator
Fri 05 Aug 2016 04:22:54 PM UTC, comment #72: 

John: Very minor point. If you run
_opengl_info_ ()
before doing any graphivs commands it returns:

warning: _opengl_info_: can not obtain OpenGL information
warning: called from
    _opengl_info_ at line 64 column 5

but after a command like plot(1:10)
it displays all the OpenGL information.

It might be good to clarify this behavior or make
a change so that it returns the open_gl information
anyhow,  if this is easy to do.

In any case, this will be very useful for the many
complex cases of graphics not sdoing what every one wants!

Michael Godfrey <godfrey>
Group Member
Fri 05 Aug 2016 03:43:33 PM UTC, comment #71: 

John D:  I checked in a modified version of your changeset here:

http://hg.savannah.gnu.org/hgweb/octave/rev/8a456af24e6b

Since I expect this function to mostly be used for debugging purposes, I renamed it to _opengl_info_.  I also used nargout to determine whether to return a struct or print the info.  It also now displays the list of extensions when printing the info.

John W. Eaton <jwe>
Group administrator
Sun 17 Jul 2016 09:07:36 PM UTC, comment #70: 

@Rik: for opengl info, how about this (opengl function) - it gets the info from the currently opened figure (or opens one if it has to)

Qt only I think as using the renderer.

No function added yet for osmesa.



(file #37929)

John Donoghue <lostbard>
Group Member
Fri 15 Jul 2016 01:58:22 PM UTC, comment #69: 

@Dmitri, comment #61:
I have run my test script again on the machine that originally produced faulty files, this time using Octave 4.0.3 from ftp://ftp.gnu.org/gnu/octave/windows/ . Lines, axes and labels are fine in the created files, apart from bug #31596 and bug #48387.


Martin Kunz <mkunz>
Wed 13 Jul 2016 12:05:35 AM UTC, comment #68: 

@Rik

I got a package working that provided the basic opengl working, however it only worked for windows and only for the Qt toolkit

Currently it creates a QGlWidget the same as the QT graphics toolskit does in order to get the opengl info as it needs a opengl window to get the information.

I think perhaps, that the toolkit needs to have some hooks into the opengl rendering that can be used  to capture the information instead, but I hadn't started looking into that as of yet. I can take a look if you want.

I hadn't yet looked at the osmesa info either, but probally the same issue there



John Donoghue <lostbard>
Group Member
Tue 12 Jul 2016 05:36:37 PM UTC, comment #67: 

Looking again at some of the entries, this looks to be a different issue in which the EPS file is generated but internally is incorrect, sorry.  The one I was referring to is the crash when there is no GUI active:


  GEN      doc/interpreter/voronoi.eps
panic: Segmentation fault -- stopping myself...


Dan Sebald <sebald>
Tue 12 Jul 2016 05:16:24 PM UTC, comment #66: 

@Dan: Did you mean to post your notes to a different bug report?  It sounds like you are having issues with the build system, rather than this report which is about Qt figures.

Rik <rik5>
Group administrator
Tue 12 Jul 2016 05:05:37 PM UTC, comment #65: 

This has been a problem for ages for me, and it's quite annoying.

Here's a new wrinkle.  Some kind of stamp file in the source tree that must have been placed there from the build process:

make[2]: Entering directory `/home/sebald/matlab/octave/test_build'
HG-ID is unchanged
make[3]: Entering directory `/home/sebald/matlab/octave/test_build'
make[3]: Leaving directory `/home/sebald/matlab/octave/test_build'
cp: cannot create regular file ‘../octave/doc/interpreter/stamp-vti’: Permission

The permission issue is from the fact that I typically build as su/root in some directory that is not under backup because I don't want the huge bundle of object files saved to a backup drive.  So I thought I would try building from the account where I keep the source code and make modifications for changesets, but then got the error.

After deleting that stamp file, compilation eventually completes, and then:


  MAKEINFO ../octave/doc/interpreter/octave.info
HG-ID is unchanged
make[3]: Entering directory `/home/sebald/matlab/octave/test_build'
make[3]: Leaving directory `/home/sebald/matlab/octave/test_build'
cp: cannot create regular file ‘../octave/doc/liboctave/stamp-1’: Permission denied
make[2]: *** [../octave/doc/liboctave/stamp-1] Error 1
make[2]: Leaving directory `/home/sebald/matlab/octave/test_build'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/home/sebald/matlab/octave/test_build'
make: *** [all] Error 2


OK, so that's another stamp file left over from a root build.  I delete that and eventually:


Installing test/.gdbinit from version at ../octave/etc/gdbinit
Installing .gdbinit from version at ../octave/etc/gdbinit
  GEN      doc/interpreter/voronoi.eps
panic: Segmentation fault -- stopping myself...
attempting to save variables to 'octave-workspace'...
save to 'octave-workspace' complete
octave exited with signal 11
  GEN      doc/interpreter/triplot.eps
panic: Segmentation fault -- stopping myself...
attempting to save variables to 'octave-workspace'...
save to 'octave-workspace' complete
octave exited with signal 11
  GEN      doc/interpreter/griddata.eps
panic: Segmentation fault -- stopping myself...
attempting to save variables to 'octave-workspace'...
save to 'octave-workspace' complete
octave exited with signal 11
  GEN      doc/interpreter/convhull.eps
panic: Segmentation fault -- stopping myself...
attempting to save variables to 'octave-workspace'...
save to 'octave-workspace' complete
octave exited with signal 11
  GEN      doc/interpreter/delaunay.eps
panic: Segmentation fault -- stopping myself...
attempting to save variables to 'octave-workspace'...
save to 'octave-workspace' complete
octave exited with signal 11
  GEN      doc/interpreter/inpolygon.eps
panic: Segmentation fault -- stopping myself...
attempting to save variables to 'octave-workspace'...
save to 'octave-workspace' complete
octave exited with signal 11
  GEN      doc/interpreter/interpft.eps
panic: Segmentation fault -- stopping myself...
attempting to save variables to 'octave-workspace'...
save to 'octave-workspace' complete
octave exited with signal 11
  GEN      doc/interpreter/interpn.eps
panic: Segmentation fault -- stopping myself...
attempting to save variables to 'octave-workspace'...
save to 'octave-workspace' complete
octave exited with signal 11
  GEN      doc/interpreter/interpderiv1.eps
panic: Segmentation fault -- stopping myself...
attempting to save variables to 'octave-workspace'...
save to 'octave-workspace' complete
octave exited with signal 11
  GEN      doc/interpreter/interpderiv2.eps
panic: Segmentation fault -- stopping myself...
attempting to save variables to 'octave-workspace'...
save to 'octave-workspace' complete
octave exited with signal 11
  GEN      doc/interpreter/plot.eps
panic: Segmentation fault -- stopping myself...
attempting to save variables to 'octave-workspace'...
save to 'octave-workspace' complete
octave exited with signal 11
  GEN      doc/interpreter/hist.eps
panic: Segmentation fault -- stopping myself...
attempting to save variables to 'octave-workspace'...
save to 'octave-workspace' complete
octave exited with signal 11
  GEN      doc/interpreter/errorbar.eps
panic: Segmentation fault -- stopping myself...
attempting to save variables to 'octave-workspace'...
save to 'octave-workspace' complete
octave exited with signal 11
  GEN      doc/interpreter/polar.eps
panic: Segmentation fault -- stopping myself...
attempting to save variables to 'octave-workspace'...
save to 'octave-workspace' complete
octave exited with signal 11
  GEN      doc/interpreter/mesh.eps
panic: Segmentation fault -- stopping myself...
attempting to save variables to 'octave-workspace'...
save to 'octave-workspace' complete
octave exited with signal 11
  GEN      doc/interpreter/plot3.eps
panic: Segmentation fault -- stopping myself...
attempting to save variables to 'octave-workspace'...
save to 'octave-workspace' complete
octave exited with signal 11
  GEN      doc/interpreter/extended.eps
panic: Segmentation fault -- stopping myself...
attempting to save variables to 'octave-workspace'...
save to 'octave-workspace' complete
octave exited with signal 11
  GEN      doc/interpreter/gplot.eps
panic: Segmentation fault -- stopping myself...
attempting to save variables to 'octave-workspace'...
save to 'octave-workspace' complete
octave exited with signal 11
  GEN      doc/interpreter/grid.eps
panic: Segmentation fault -- stopping myself...
attempting to save variables to 'octave-workspace'...
save to 'octave-workspace' complete
octave exited with signal 11
  GEN      doc/interpreter/spmatrix.eps
panic: Segmentation fault -- stopping myself...
attempting to save variables to 'octave-workspace'...
save to 'octave-workspace' complete
octave exited with signal 11
  GEN      doc/interpreter/spchol.eps
panic: Segmentation fault -- stopping myself...
attempting to save variables to 'octave-workspace'...
save to 'octave-workspace' complete
octave exited with signal 11
  GEN      doc/interpreter/spcholperm.eps
panic: Segmentation fault -- stopping myself...
attempting to save variables to 'octave-workspace'...
save to 'octave-workspace' complete
octave exited with signal 11
  GEN      doc/interpreter/splinefit1.eps
panic: Segmentation fault -- stopping myself...
attempting to save variables to 'octave-workspace'...
save to 'octave-workspace' complete
octave exited with signal 11
  GEN      doc/interpreter/splinefit2.eps
panic: Segmentation fault -- stopping myself...
attempting to save variables to 'octave-workspace'...
save to 'octave-workspace' complete
octave exited with signal 11
  GEN      doc/interpreter/splinefit3.eps
panic: Segmentation fault -- stopping myself...
attempting to save variables to 'octave-workspace'...
save to 'octave-workspace' complete
octave exited with signal 11
  GEN      doc/interpreter/splinefit4.eps
panic: Segmentation fault -- stopping myself...
attempting to save variables to 'octave-workspace'...
save to 'octave-workspace' complete
octave exited with signal 11
  GEN      doc/interpreter/splinefit6.eps
panic: Segmentation fault -- stopping myself...
attempting to save variables to 'octave-workspace'...
save to 'octave-workspace' complete
octave exited with signal 11
  TEXI2DVI doc/interpreter/octave.dvi
/usr/bin/texi2dvi: etex exited with bad status, quitting.
make[2]: *** [doc/interpreter/octave.dvi] Error 1
make[2]: Leaving directory `/home/sebald/matlab/octave/test_build'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/home/sebald/matlab/octave/test_build'
make: *** [all] Error 2


So, "make install" won't complete.

If I were to make a change and recompile, the <buildtree>/run-octave executable works but there is this slow process of trying to build all these doc files again.  Generally, re-compilation has become so slow that it is difficult to tackle significant C++ code mods, for me anyway.  Tweaking a line here or there is do-able, but other than that, one spends most of the time waiting for compilation to complete.  About a year ago it used to be the case that once compilation got to the GUI subdirectory the compiler would speed through those files--so making C++ code changes in the GUI wasn't too bad.  But now too the GUI files are compiled at a crawl.  Is there anything that can be done to speed compilation as well?

Dan Sebald <sebald>
Tue 12 Jul 2016 04:12:05 PM UTC, comment #64: 

@JohnD: I think both of your suggestions in comment #56 are good ideas:

1) A --softwareopengl option to octave at the command line.
2) A function opengl to get information about the current installation.

At least for 4.2, I think option 2 is required.  We need more information before we can debug these reports.  Consider also, if possible, whether you can get information about OSMESA since that seems also to be important.

This bug can stay a blocker until the opengl() function is written, and then downgraded.





Rik <rik5>
Group administrator
Tue 12 Jul 2016 11:11:49 AM UTC, comment #63: 

Michael, I agree entirely with your last paragraph.

Could someone with authority please clarify exactly what the blocking issue is, and whether it is still a blocker or if that is just left over from before the patch of comment #30.

Lachlan Andrew <lachlan>
Tue 12 Jul 2016 07:56:30 AM UTC, comment #62: 

The ssh issue referred to below definitely depends on
both of the 2 systems involved. But, some combinations
which run glxgears and other X11 tests correctly do not
run Octave plots (usuallly hangs). On the other hand
some combinations currently do not run glxgears, so the
faults are not all due to Octave.

I would be better to keep this and the print issue
separate. And, it seems unrealistic to consider the
ssh hang issue to be a blocker.

Michael Godfrey <godfrey>
Group Member
Tue 12 Jul 2016 06:52:30 AM UTC, comment #61: 

Do we still have printing issue? The original report is more than a year old and its for 4.0.0rc.
Figs in doc are now built with qt backend and apparently it is going ok for majority of builders.

Dmitri.
--

Dmitri A. Sergatskov <dasergatskov>
Tue 12 Jul 2016 06:35:43 AM UTC, comment #60: 

Thanks for that data point, Dmitri.

It sounds like you are saying that the ssh issue isn't an Octave issue.  However, the blocker here is the (in)ability to print to a file on a single computer.  That does seem to be an Octave problem -- at least inasmuch as we need to work around any limitations of OpenGL so that printing is possible somehow.

Lachlan Andrew <lachlan>
Tue 12 Jul 2016 06:03:47 AM UTC, comment #59: 

It does not look like an octave problem.
If I ssh from i7 (HD3000 graphics) to amd (radeon graphics), OpenGL program would not work (even glxinfo just returns
name of dislay: localhost:10.0
and then hangs -- Ctr-C returns to prompt).

Going the other way (ssh from amd to i7 works.
running e.g. sombrero (with qt graphics) results in

libGL error: failed to authenticate magic 1
libGL error: failed to load driver: r600

error messages on the octave console, but sombrero
diplays just fine.
The same errors I get if I run glxgears...

Dmitri.
--

Dmitri A. Sergatskov <dasergatskov>
Tue 12 Jul 2016 05:27:24 AM UTC, comment #58: 

John, have you got a version of  opengl  that is ready to share?

This bug is marked as a blocker for 4.2.0, and it seems that it will be tricky to track down.

Thanks!

Lachlan Andrew <lachlan>
Wed 23 Mar 2016 10:09:14 PM UTC, comment #57: 

Probably do need something like the opengl command to find out what is going wrong on failing machines.  It should probably be a separate Feature Request.  It may also need to include OSMESA information.

Rik <rik5>
Group administrator
Wed 23 Mar 2016 04:26:07 PM UTC, comment #56: 

Since we are now using software rendering for opengl in QT, is that the issue with some reports of opengl running very slowly?

Since the reason it was used is that some opengl drivers appeared to cause issues with figures, maybe we should add a command line option to allow enable/disable of the opengl software rendering similar to matlab with the -softwareopengl option [1]

On a side note, I had started writing a oct file that displays the opengl info similar to the matlab opengl function (the info and data options) - maybe usefully for helping identify what drivers are causing issues.

Running it in windows 7 with octave 4.0.1-rc4:

>> opengl("info")
version  = 1.1.0
vendor   = Microsoft Corporation
renderer = GDI Generic
# of extensions = 3


[1] http://www.mathworks.com/help/matlab/ref/opengl.html



John Donoghue <lostbard>
Group Member
Tue 02 Jun 2015 04:48:06 PM UTC, comment #55: 

Retagging release from 4.0.0-rc3 to 4.0.0.

John W. Eaton <jwe>
Group administrator
Tue 12 May 2015 06:26:30 PM UTC, comment #54: 

This:

2. For Fedora systems, using the environment variable
   which prevents hangs and visible off fpr print, works
   perfectly well.

should have read:

2. For Fedora systems, using the environment variable
   which prevents hangs and visible off, for print, fltk works
   perfectly well.

Sorry for typo.

Michael Godfrey <godfrey>
Group Member
Tue 12 May 2015 04:39:18 PM UTC, comment #53: 

I assume that this is the surviving bug report and that
44915 will be merged into it.

from the testing that I have done:

1. gnuplot still "works." I do not find it adequate,  but
   it does some things OK.  And, it appears to be less
   subject to system-dependence.

2. For Fedora systems, using the environment variable
   which prevents hangs and visible off fpr print, works
   perfectly well.

3. qt has problems -- worse when used through ssh.

Matlab seems to have been struggling with similar problems.

So, for the 4.0 release, would it make sense to:

1. Set default graphics_toolkit('fltk')

2. Recommend trying qt or reverting to gnuplot if
   necessary.

More testing of fltk on other systems will be needed to
validate this approach. However, in any case, it appears
that quite a lot of work will be needed to get qt working
as well as fltk.

Of course, this is a "view from Fedora!"

Michael Godfrey <godfrey>
Group Member
Tue 12 May 2015 03:54:12 PM UTC, comment #52: 

John,

I am building with current FC21 which appears to be
qt5. rpm -qa|grep qt<3,4,5>
shows only qt5 and:
qt3-3.3.8b-63.fc21.x86_64
which apparently is needed by some other package.

You said:

>So on some systems the Qt graphics window hangs when connecting >with ssh but we have no clue why or how to avoid that problem? >But the fltk widget works?


Yes, fltk works completely as long as print is only done
with visible off.  I do think that it is a good idea to only
run the print code with visible off in any case. This is
more efficient, allows use when visible on would not work,
and avoids distracting flicker on the screen. So, I would
advocate pushing a patch for that separate from trying to
fix qt.

From what I see, there is something basically wrong in the
qt code. The fact that randomly incomplete plots show up
when running qt through ssh suggests that there is a
race condition/timing problem at least.

Anything else I can provide?


Michael Godfrey <godfrey>
Group Member
Tue 12 May 2015 01:16:33 PM UTC, comment #51: 

Michael:  Sorry, I missed your comment (#50 (!)).

Are you actually building with qt5?  I thought we bypassed that even if it was present on the build system and were still using qt4.

Yes, the --no-gui option still uses the octave-gui binary that is linked with Qt but it just doesn't start the GUI.

So on some systems the Qt graphics window hangs when connecting with ssh but we have no clue why or how to avoid that problem?  But the fltk widget works?

Ugh.  This is really becoming annoying.  I have little to no clue how to fix these problems.


John W. Eaton <jwe>
Group administrator
Thu 07 May 2015 10:27:12 PM UTC, comment #50: 

John,
I read that qt bug report as saying it was fixed in 5.3
but my qt5is at 5.4.

But, I also noticed that the fix you mentioned only caused octave-gui to be rebuilt. If I use
octave --no-gui does this end up using octave-qui ?
And, should octave have been rebuilt anyhow?



Michael Godfrey <godfrey>
Group Member
Thu 07 May 2015 08:42:40 PM UTC, comment #49: 

Yes, that is the revision ID of my change, so if you are using that, you have it.

I did a little digging and it looks like the problem is this, or something similar:

https://bugreports.qt.io/browse/QTBUG-37878

I found it crashing inside the same Qt function anyway...

John W. Eaton <jwe>
Group administrator
Thu 07 May 2015 08:01:23 PM UTC, comment #48: 

This is using stable:
stable                     20168:17542843a20b

does this have your patch?

Michael Godfrey <godfrey>
Group Member
Thu 07 May 2015 06:07:06 PM UTC, comment #47: 

John:  ssh still hangs. But, since this appears to have been
true for some time I would still say that it should not be
a blocker for 4.0.

But, it and related print problems should be given serious
attention after the release.

I do think that relying on Opengl is risky. Maybe this
should be made optional. My system with Intel HD 6000
graphics shows that the software, at least for Fedora,
lags behind the hardware availability.



Michael Godfrey <godfrey>
Group Member
Thu 07 May 2015 03:42:16 PM UTC, comment #46: 

Ugh, testing this with my builds for RHEL 6 and SuSE 11 I noticed that it works properly if I am logging in directly but not when I connect with SSH (with X forwarding enabled).  Over the SSH connection Octave crashes as soon as I try to plot something.  As I recall, X forwarding workd in the past.  How strange.

Michael, if you have tried my patch on a Fedora system, what do you see when connecting with SSH?

This is getting to be a real PITA.

John W. Eaton <jwe>
Group administrator
Wed 06 May 2015 09:03:45 PM UTC, comment #45: 

This looks good enough so that this bug should no
longer be a blocker.

Getting all the cases straight and fixing the various
other problems with plotting (remote access, hardware
use, etc) will take quite a while.

From my standpoint (mainly Fedora...) the current rc4+John's patch
is much better than previous systems.


Michael Godfrey <godfrey>
Group Member
Wed 06 May 2015 07:23:26 PM UTC, comment #44: 

Oh, Windows doesn't use Mesa?  Well debugging that is out then.

Dan Sebald <sebald>
Wed 06 May 2015 07:19:18 PM UTC, comment #43: 

Identifying the driver/version for the systems that are failing would help.  For Linux, glxinfo appears to have the GL_ parameters.

The source tree for the various drivers is available on line in git, so this bug could be searched there.

http://cgit.freedesktop.org/mesa/mesa/

It's possible to build all the drivers and Mesa locally to find out if the latest drivers fix the problem (or help debug the driver).  This takes some time, not too bad but also not something for the impatient.  It's a lot of this sort of compiling:

[compiling resources so that they look to a different package configuration than the OS bundles default package directory...the install will place the library files and drivers in the local folder, not system folder]


git clone <repository>
PKG_CONFIG_PATH=/home/local/lib/pkgconfig:/usr/local/lib64:$PKG_CONFIG_PATH cmake .
make
make install


and

[Run the application so that it loads drivers from the new library path]


LD_LIBRARY_PATH=/home/local/lib:/usr/local/lib64 <path>/run-octave


The end result is a bunch of library files in local/lib, and a half dozen drivers in local/lib/dri.  Windows might be more challenging.


Dan Sebald <sebald>
Wed 06 May 2015 07:05:51 PM UTC, comment #42: 

Rik: Yes, that's the result that the person doing the testing for me saw.  FLTK produced the same incorrect EPS file but it was correct with Qt.

I will look at making it optional.

John W. Eaton <jwe>
Group administrator
Wed 06 May 2015 07:00:47 PM UTC, comment #41: 

Won't the change in comment #30 only apply to Qt-based plotting?  If it is correct that the hardware driver for OpenGL that is causing the problem then FLTK should still produce faulty output.

This could, at least, help confirm that the issue is the HW driver.  Steps to take are running the altered code from jwe in comment #30 on a failing system and then printing the failing figure twice using the two toolkits qt and fltk.  The qt toolkit should pass and the FLTK toolkit should fail.

Assuming that is correct, it would be nice to have something like mesa installed so that we could have both toolkits switch over to software rendering if there was a problem.  It certainly would also be good to have a configuration option for this at run-time.  It's not like we have gotten a lot of reports about this so HW based rendering is probably okay on > 75% of machines.  The majority shouldn't be held hostage to the minority in this case.

Rik <rik5>
Group administrator
Wed 06 May 2015 06:36:27 PM UTC, comment #40: 

For now I think I'll just apply that change, but perhaps we should consider making it optional?  It would be great to be able to detect the bug and use indirect rendering in those cases, but I don't know how we would do that.

John W. Eaton <jwe>
Group administrator
Wed 06 May 2015 06:34:59 PM UTC, comment #39: 

I now have a report that the change I posted in comment #30 fixes (or at least avoids) the problem.  So I suspect some kind of driver or hardware bug, which would also seem to explain why the bug appears on some systems and not others.

John W. Eaton <jwe>
Group administrator
Wed 06 May 2015 05:25:33 PM UTC, comment #38: 

I've plotted the points you've indicated.  See attached PNG file.  (Screenshot is better quality than the printed output.)

It looks as though the second data point is the one that is missing.  In this group of data, I've donated by ** the lines that seem incorrect:


   LINE: 119.519989 82.727989 (0.000000 0.000000 1.000000)
   LINE: 119.519989 82.727989 (0.000000 0.000000 1.000000)  **
   LINE: 208.800003 153.143997 (0.000000 0.000000 1.000000) **
   LINE: 208.800003 153.143997 (0.000000 0.000000 1.000000)
   LINE: 208.800003 153.143997 (0.000000 0.000000 1.000000)
   LINE: 253.440002 188.351990 (0.000000 0.000000 1.000000)
   LINE: 253.440002 188.351990 (0.000000 0.000000 1.000000)


In other words it seems the two highlighted lines should be replaced by a point that is missing.  Something like:


   LINE: 119.519989 82.727989 (0.000000 0.000000 1.000000)
   LINE: 163.500000 117.500000 (0.000000 0.000000 1.000000) ++
   LINE: 163.500000 117.500000 (0.000000 0.000000 1.000000) ++
   LINE: 208.800003 153.143997 (0.000000 0.000000 1.000000)
   LINE: 208.800003 153.143997 (0.000000 0.000000 1.000000)
   LINE: 253.440002 188.351990 (0.000000 0.000000 1.000000)
   LINE: 253.440002 188.351990 (0.000000 0.000000 1.000000)


Could be an OpenGL driver bug, don't know.

Was this data from the very first time the print was attempted?  It's not the case that the print was done prior and the second time the alloc'ed gl2ps feedback buffer was a slightly offset memory location, is it?  (Trying to discern if the ** lines above were actually written into memory or were whatever happened to be in memory, which came from a previous print.)


Dan Sebald <sebald>
Wed 06 May 2015 04:02:21 PM UTC, comment #37: 

When looking at printing plot (1:10), I saw the following differences in the line.  On the bad system, it would start at (2,2) or (3,3) and on my virtualbox VM, it would start at (1,1) as expected.  I've trimmed the output to just focus on the line segment, not the axes.

The following lines correspond to the endpoints of the 9 line segments from 1:10 (RGB value is 0 0 1, so that's the blue line).  These are the values that gl2ps is getting back from OpenGL.  Note that the first two segments are duplicate points, so they are don't end up being line segments.  This explains why part of the line is missing.

The bad system:


   LINE: 119.519989 82.727989 (0.000000 0.000000 1.000000)
   LINE: 119.519989 82.727989 (0.000000 0.000000 1.000000)
   LINE: 208.800003 153.143997 (0.000000 0.000000 1.000000)
   LINE: 208.800003 153.143997 (0.000000 0.000000 1.000000)
   LINE: 208.800003 153.143997 (0.000000 0.000000 1.000000)
   LINE: 253.440002 188.351990 (0.000000 0.000000 1.000000)
   LINE: 253.440002 188.351990 (0.000000 0.000000 1.000000)
   LINE: 298.079987 223.559998 (0.000000 0.000000 1.000000)
   LINE: 298.079987 223.559998 (0.000000 0.000000 1.000000)
   LINE: 342.720001 258.768005 (0.000000 0.000000 1.000000)
   LINE: 342.720001 258.768005 (0.000000 0.000000 1.000000)
   LINE: 387.359985 293.976013 (0.000000 0.000000 1.000000)
   LINE: 387.359985 293.976013 (0.000000 0.000000 1.000000)
   LINE: 432.000000 329.183990 (0.000000 0.000000 1.000000)
   LINE: 432.000000 329.183990 (0.000000 0.000000 1.000000)
   LINE: 476.639984 364.391998 (0.000000 0.000000 1.000000)
   LINE: 476.639984 364.391998 (0.000000 0.000000 1.000000)
   LINE: 521.280029 399.600006 (0.000000 0.000000 1.000000)


On my system, I see the following:


   LINE: 116.203125 92.433594 (0.000000 0.000000 1.000000)
   LINE: 159.601563 126.660156 (0.000000 0.000000 1.000000)
   LINE: 159.601563 126.660156 (0.000000 0.000000 1.000000)
   LINE: 203.000000 160.890625 (0.000000 0.000000 1.000000)
   LINE: 203.000000 160.890625 (0.000000 0.000000 1.000000)
   LINE: 246.402344 195.121094 (0.000000 0.000000 1.000000)
   LINE: 246.402344 195.121094 (0.000000 0.000000 1.000000)
   LINE: 289.800781 229.351563 (0.000000 0.000000 1.000000)
   LINE: 289.800781 229.351563 (0.000000 0.000000 1.000000)
   LINE: 333.199219 263.578125 (0.000000 0.000000 1.000000)
   LINE: 333.199219 263.578125 (0.000000 0.000000 1.000000)
   LINE: 376.597656 297.808594 (0.000000 0.000000 1.000000)
   LINE: 376.597656 297.808594 (0.000000 0.000000 1.000000)
   LINE: 420.000000 332.039063 (0.000000 0.000000 1.000000)
   LINE: 420.000000 332.039063 (0.000000 0.000000 1.000000)
   LINE: 463.398438 366.265625 (0.000000 0.000000 1.000000)
   LINE: 463.398438 366.265625 (0.000000 0.000000 1.000000)
   LINE: 506.796875 400.496094 (0.000000 0.000000 1.000000)


John W. Eaton <jwe>
Group administrator
Wed 06 May 2015 03:54:17 PM UTC, comment #36: 

If you do a binary comparison of the buffer, I'd expect lots of differences becuase I wouldn't expect any two implementations to return bit-for-bit identical results.  Doing something like what I did in the patch I posted earlier for gl2ps would probably be easier to decipher and you could easily ignore any minor differences.

https://savannah.gnu.org/bugs/download.php?file_id=33932

John W. Eaton <jwe>
Group administrator
Wed 06 May 2015 03:40:43 PM UTC, comment #35: 

OK.  Can't rule out multiple issues happening that might be confusing matters.

To get some insight, it might be useful to compare the contents of the feedback buffer between a properly working system (wouldn't have to be any particular kind, I think).  That is, after calling


         opengl_renderer::draw (go);


place a short hunk of code that sends the contents of

gl2ps->feedback

to a binary file.  One can then do a binary comparison between feedback data for a working system and non-working system--sort of pick apart what may be happening.  If the binary data compares exactly, then it could be that gl2ps has a bug or is using an incompatible header definition for the API.  If the binary data doesn't compare exactly, then one could open the files in a binary editor and search for the hex equivalents to the tokens that OpenGL is using.  Does that sound like more work than the help it would yield?

[
This is an aside, but I think that this feedback mode really could be the source of the temporary flash of colors I see.  In the gl2ps code I only see the one instance where the render mode is changed back to GL_RENDER.  I.e.,


static GLint gl2psPrintPrimitives(void)
{
  GL2PSbsptree *root;
  GL2PSxyz eye = {0.0F, 0.0F, 100.0F * GL2PS_ZSCALE};
  GLint used;

  used = glRenderMode(GL_RENDER);

  if(used < 0){
    gl2psMsg(GL2PS_INFO, "OpenGL feedback buffer overflow");
    return GL2PS_OVERFLOW;
  }

  if(used > 0)
    gl2psParseFeedbackBuffer(used);


What happens if during the time that the Qt figure window attempts to redraw the figure?  At least the feedback buffer is cleared from memory after the render mode is turned back to GL_RENDER.  Otherwise doing an asynchronous replot in a different thread would be a problem...  BTW, it might be useful to print out the value of "used" in the above gl2ps code.
]

The gl2ps documentation could be a tad clearer on what it is doing.  I mean, now I understand more having seen the innards.  However, the statement about gl2ps culling primitives is not an accurate one.  The culling actually happens prior to gl2ps and gl2ps has culling simply as a matter of consequence.  Here's documentation on the glFeedbackBuffer routine:

https://msdn.microsoft.com/en-us/library/windows/desktop/dd373532(v=vs.85).aspx

http://www.glprogramming.com/red/chapter13.html

Note that

"
Each block begins with a code indicating the primitive type, followed by values that describe the primitive's vertices and associated data. The glFeedbackBuffer function also writes entries for bitmaps and pixel rectangles. Feedback occurs after polygon culling and glPolygonMode interpretation of polygons has taken place, so polygons that are culled are not returned in the feedback buffer. It can also occur after polygons with more than three edges are broken up into triangles, if the OpenGL implementation renders polygons by performing this decomposition.
"

All right that pretty much confirms that the primitives are culled before reaching gl2ps.

FWIW, if intricate debugging is required, it's possible to put some marker tokens in the feedback buffer data using glPassThrough:

"
You can insert a marker into the feedback buffer with glPassThrough.
"

OK, so why is OpenGL culling lines when sent to the feedback buffer as opposed to one of the video drivers?  Is there something configuring the image size that is supposed to be used by OpenGL for the feedback buffer?  Some other obscure setting?

Dan Sebald <sebald>
Wed 06 May 2015 02:43:08 PM UTC, comment #34: 

Dan, I have a report that the bug happens with a copy of 3.8.2 that I built using the current mxe-octave build tools and dependencies.  So as I understand it, there is no threading there (at least not that we are adding into the mix) because all FLTK event processing happens when readline is idle, which is in the same thread as the Octave interpreter.

What's weird is that the same person reports that an older build of 3.8.2 that I made does not show the problem.  I have no clue why that would be.

John W. Eaton <jwe>
Group administrator
Wed 06 May 2015 02:30:55 PM UTC, comment #33: 

OK, one question is whether this is a strictly Qt issue.  I was under that impression, but I'm not certain if there's even a distinction.  Both Qt and fltk use the _opengl_print_.m command.  So, even though Qt and fltk are different frameworks, it may be the environment of the executing program that influences the behavior.  And that environment may be attributed to Qt in the case of GUI operation and --no-gui operation.  In terms of environment, --no-gui and normal operation are the same.  It's the --no-gui-libs option which results in a different underlying environment.

Question:  When you say you are seeing the same behavior in FLTK as in Qt, is that using the --no-gui-libs option?

Dan Sebald <sebald>
Wed 06 May 2015 06:38:04 AM UTC, comment #32: 

@John (comment #26): Yes, I do see the same strange results with fltk. All the submitted image files that start with 'f' have been produced by fltk.

I can try the builds of the older versions, but not before tomorrow, sorry.

Martin Kunz <mkunz>
Wed 06 May 2015 06:35:50 AM UTC, comment #31: 

Oh, I see.  Looking at the patch https://savannah.gnu.org/support/download.php?file_id=33932 I now have a little better understanding of the way gl2ps works.  There is a possibility this is a thread timing issue.  Not 100% sure, but I'll explain.

Contrary to what I was thinking, the gl2ps code is not operating directly on shared memory in threads.  There is some kind of feedback feature in OpenGL that gl2ps is using to get a copy of the plot contents in a memory buffer.  gl2ps then operates on that feedback buffer.

In gl2ps-renderer.cc, the general sequence of events is


  gl2psBeginPage();
  old_print_cmd = print_cmd;
  opengl_renderer::draw(go);
  glFinish();


It's inside gl2psBeginPage where the acquisition of the data occurs:


  gl2ps->feedback = (GLfloat*)gl2psMalloc(gl2ps->buffersize * sizeof(GLfloat));
  glFeedbackBuffer(gl2ps->buffersize, GL_3D_COLOR, gl2ps->feedback);
  glRenderMode(GL_FEEDBACK);


There is a buffer created, that buffer is set up as the feedback buffer OpenGL will use, then opengl_renderer::draw(go) fills in that memory--at least that is how I understand it.  However, isn't it the case that opengl_renderer could have some element of it that has main thread affinity, not the interpreter thread?  If so, something in the opengl_renderer::draw chain of events could end up queued across threads in Qt.  Then there will be a race condition.  The ::draw will be filling in the contents of gl2ps->feedback while the code in the interpreter thread continues onward.  My guess is that most of the time Martin sees random graphics because that feedback buffer is yet to be filled when gl2ps operates on it.  However, it may be the case that once in a while ::draw finishes some of the plot before gl2ps operates on it, but it is never the complete plot, i.e., the elements we've discussed as being dropped aren't in the feedback buffer yet.

It seems to me that in the following code there might need to be a semaphore wait as noted by []:


          if (ret == GL2PS_ERROR)
            {
              old_print_cmd.clear ();
              error ("gl2ps-renderer::draw: gl2psBeginPage returned GL2PS_ERROR");
              return;
            }

          old_print_cmd = print_cmd;

          opengl_renderer::draw (go);

 [Need some kind of Mutex handshake right here to make ]
 [sure the above operation has completed if the draw is]
 [being done in the main thread.                       ]

          // Without glFinish () there may primitives be missing in the
          // gl2ps output.
          glFinish ();


Just my thoughts.  There is a possibility this could be the source of the strange graphics I see when printing in Qt.  That is, Qt redraw/rendering while in GL_FEEDBACK mode maybe, perhaps?  Maybe that has a strange effect.  Don't know.

Dan Sebald <sebald>
Wed 06 May 2015 04:20:49 AM UTC, comment #30: 

Could someone who is able to reproduce this bug please try the following version:

http://jweaton.org/octave-2015-05-06-00-01.zip

It was built with this change in an attempt to enable software rendering:


diff --git a/libgui/graphics/GLCanvas.cc b/libgui/graphics/GLCanvas.cc
--- a/libgui/graphics/GLCanvas.cc
+++ b/libgui/graphics/GLCanvas.cc
@@ -38,7 +38,8 @@ namespace QtHandles
 {

 GLCanvas::GLCanvas (QWidget* xparent, const graphics_handle& gh)
-  : QGLWidget (QGLFormat(QGL::SampleBuffers), xparent), Canvas (gh)
+  : QGLWidget (QGLFormat (QGL::SampleBuffers | QGL::IndirectRendering),
+               xparent), Canvas (gh)
 {
   setFocusPolicy (Qt::ClickFocus);
 }


John W. Eaton <jwe>
Group administrator
Wed 06 May 2015 03:48:33 AM UTC, comment #29: 

Dan, I modified gl2ps so that it prints the list of verices as it gathers them from the OpenGL buffer.  The vertices are already screwed up at that point.  So it seems unlikely that gl2ps is to blame, though I suppose it is still possible somehow.  I'm attaching my gl2ps changes in case anyone else would like to experiment with them and maybe get some clue about what could be happening.

(file #33932)

John W. Eaton <jwe>
Group administrator
Tue 05 May 2015 09:26:57 PM UTC, comment #28: 

Thanks for trying the green background.  I think it is good to just confirm what is happening before searching for bigger problems.  Well, definitely elements of the plot are being dropped.  That's confirmed.  I loaded one of the SVG plots into inkscape and ungrouped the plot.  Individual plot elements are used, not some big bitmap.

Threading is always a concern, but in this case I don't know if that is a problem because the gl2ps/OSMesa code may be running in the same thread as Octave core.  (Qt graphics certainly is not in the same thread.)  I've looked a bit and it seems _osmesa_print_() is being run from the worker Octave interpreter thread.

What complicates this is that gl2ps does more than just translate the OpenGL commands to some output format, i.e., a simple driver.  Here is what the doc indicates:

"
The main difference between GL2PS and other similar libraries (see section 7) is the use of sorting algorithms capable of handling intersecting and stretched polygons, as well as non manifold objects. GL2PS provides advanced smooth shading and text rendering, culling of invisible primitives, mixed vector/bitmap output, and much more...
"

Culling of invisible primitives makes me think that gl2ps could be the code that is dropping the axis line and one of the triangles from the background.  Of course, it is mistakenly doing so, if that is the case.  What configuration on Octave's side would make gl2ps think some primitive is hidden when it isn't?

Let's go back to the thread hypothesis.  The way that the random bug could occur would be a thread clash, i.e., two threads operating on the same memory simultaneously.  It doesn't seem likely (or should I say necessary?), yet on the other hand, something comes to mind which is that I've often seen the Qt terminal go into this strange state at the moment of printing in which random splotches of color appear, and when printing is complete the drawing comes back to normal.  Is it possible that a redraw is occurring in Qt at the same moment as gl2ps is working?  If so, should this even be a problem if the memory accessed is only being read and not modified?  (If it were a bus issue, sure, but I think all those sorts of bus clashes are worked out at the firmware level.)

There's lots going on here.  Hard to pin down.

Dan Sebald <sebald>
Tue 05 May 2015 06:21:28 PM UTC, comment #27: 

Hmm, well, now I have a report that my current build of 3.8.2 fails but an older version that I built around the time of the 3.8.2 release works properly.

I have no idea what dependency or build tool has changed that could cause that.

John W. Eaton <jwe>
Group administrator
Tue 05 May 2015 03:55:53 PM UTC, comment #26: 

Martin, do you also see strange results with fltk?

Can you please try the following Windows distribution and see if it also fails (using the FLTK toolkit, since that was all that was in 3.8.2).

  http://jweaton.org/octave-2015-05-04-20-31.zip

I have reports that 3.8.2 was working on systems where the current 4.0.0-rc4 builds fail.  This binary is a build of 3.8.2 from the Mercurial archive with the current build tools and two changes:

  * print.m bypasses ghostscript and just saves the PostScript generated by gl2ps in a file.

  * the zip file includes a copy of the new octave.vbs wrapper script.

Use the octave.vbs wrapper script to start Octave, then in addition to whatever tests you've been running, will you please try

  plot (1:10)
  print -depsc foo.eps

and check the generated foo.eps file?  The symptom of failure for this one is usually that the line on the screen runs from (1,1) to (10,10) but in the printed file is typically starts at (2,2) or sometimes (3,3) (I think).

Thanks.

John W. Eaton <jwe>
Group administrator
Tue 05 May 2015 03:37:07 PM UTC, comment #25: 

Response to Dan (comment #24)

I did not change settings after installation, so both machines produce plots with default settings. The axes equivalent to figure position is axis position, right?

I ran the test script again on both machines and checked the properties of figure 2:


>> get (2,"position")
ans =
   300   200   560   420
>> get (0,"defaultfigureposition")
ans =
   300   200   560   420
>> get (get (2,"children") , "position")
ans =
   0.13000   0.11000   0.77500   0.81500
>> get (0,"defaultaxesposition")
ans =
   0.13000   0.11000   0.77500   0.81500


The results are exactly the same on both systems.

The second test you proposed was not so easy to perform because, as I wrote before, the result is not deterministic: Sometimes the curve is fine, sometimes broken, sometimes it has wild lines. Same for the axes, it needed several runs until I got an EPS with green background and a missing axis. The file f2greenBG.eps was created on the "problematic" machine 1 with the following commands:


figure();
fflush(stdout);
x = 1:1000;
y = sin (x/300);
plot (x,y);
set(gca, 'color', [0 1 0])
xlabel ("Time in s")
ylabel ("\\Omega")
fprintf ("\nSaving figure 2\n");
print -dpng -r0 f2.png
print -dpdf -r0 f2.pdf
print -dsvg -r0 f2.svg
print -depsc -r0 f2.eps


After creation, I renamed the file from f2.eps to f2greenBG.eps in order to prevent it from being overwritten.

To me it seems pretty clear that the x-axis is really missing, not covered - the lower background triangle is missing and opening the EPS with Inkscape I cannot find a line for the axis.

Doesn't the randomness of the error indicate that Rik is probably right and we are facing something thread-related here, a race condition maybe? One big difference between my two systems is that the machine that produces faulty files is much faster than the other (3.3GHz with graphics card vs. 2.4GHz with onboard graphics).

(file #33925)

Martin Kunz <mkunz>
Mon 04 May 2015 09:41:09 PM UTC, comment #24: 

To make a good comparison, it seems to me that the sizes of the plots on the two systems, when printed, must be the same.  Check the sizes between the two systems, i.e.,

get(gcf, 'position')

Are they the same on the two systems?  Is there some equivalent size setting for the axis?  Seems there should be, but I don't see anything.

Here's another test.  Please change the color of the axis so that the background is all one color and generate the EPS file (and other formats if you like).  That is,

set(gca, 'color', [0 1 0])

The idea would be to check if the missing axis line of the border ("box") has actually been lost or if it is covered up by something else.  If the green background extends to the location the box line should be (have to zoom in to tell) then there is a possibility (but not certainty) the line is present but obscrued by the background "rectangle" (actually two triangles).  The the green background doesn't extend that far, then it is almost certainly that the line was dropped along the way.  Can't hazard a guess at either case because that SVG output even seems to have one of the background triangles missing.

Dan Sebald <sebald>
Mon 04 May 2015 08:38:02 PM UTC, comment #23: 

As far as I can tell, this is not a problem with gl2ps.  When gl2ps gets the vertex data from the OpenGL buffer it is already screwed up.

John W. Eaton <jwe>
Group administrator
Mon 04 May 2015 07:59:08 PM UTC, comment #22: 

FTR the original bug for missing line segments is bug #44125.

There are example .c files distributed along with gl2ps source tarball that could let us see if gl2ps itself is behaving properly on those failing machines. Many gl2ps features can be tested including sorting algorithms.
Is it possible to cross compile those files and run the binaries on windows?

Pantxo Diribarne <pantxo>
Group Member
Mon 04 May 2015 04:36:03 PM UTC, comment #21: 

@Rik (comment #18) You mean using a build that does not include Qt, right? In case you don't, I started "standard" Octave 4.0.0-rc4 with the option you proposed...


C:\Octave\Octave-4.0.0-rc4\bin>octave.exe --no-gui-libs


... and ran the test script again. The results look like before, missing curve segments and missing axes.

Martin Kunz <mkunz>
Mon 04 May 2015 04:03:10 PM UTC, comment #20: 

@Martin: I think that the fact that the line starts at (2,2) and not (1,1) is a consequence of reverting the change associated to the chosen sorting algorithm in gl2ps (NO_SORT<->SIMPLE_SORT). This change was introduced to solve precisely this kind of problem of culled lines IIRC but I can find the associated bug report ATM.

I thus see no regression between 3.8.0 and 4.0rc4-original (no reverted dll) from those tests.

Pantxo Diribarne <pantxo>
Group Member
Mon 04 May 2015 03:30:54 PM UTC, comment #19: 

Hi and sorry for the delay, I can only test this at work because on my PC at home this problem does not show up.

First I deinstalled rc3 and installed rc4 from alpha.gnu.org on both machines. Like before, machine 1 produces faulty files from fltk figures, machine 2 produces correct files.

Some graphics-related info on the two systems retrieved with GLview:
Machine 1
=========
Renderer: ATI Radeon HD 5450
Operating System: Microsoft Windows 7 Enterprise
Processor: Intel Core i3-2120 CPU @ 3.30GHz, Sandy Bridge, Family 6h, Model: 2ah, Stepping: 7h
OpenGL version: 4.2
Driver version: 9.12.0.0 Catalyst 9.012 19-Dez-12

Machine 2
=========
Renderer: Mobile Intel 4 Series Express Chipset Family
Operating System: Microsoft Windows 7 Enterprise
Processor: Intel Core 2 Duo CPU P8600 @ 2.40GHz, Penryn, Family 6h, Model: 17h, Stepping: ah
OpenGL version: 2.1
Driver version: 8.15.10.2302 11-Feb-11

(Typos possible, but I double-checked the version numbers)


I don't have Octave 3.8.2 at hand, but 3.8.0 is still installed so I ran the test script there. Guess what: I get basically the same faults in the saved files, see the attached f3_octave3.8.0.eps as an example.

After this test I switched back to 4.0.0-rc4

In the next step I replaced print.m and restarted Octave 4.0.0-rc4 as you suggested in comment #13. I do get the expected output:

+verbose+
epstool command: ' cat > "foo.eps" '
opengl-pipeline: ' cat > "foo.eps" '
-verbose-

foo.eps is attached. The axes look fine, but the line starts at (2,2), not (1,1) as one would expect!

I switched back to the original print.m and replaced liboctinterp-3.dll as suggested in comment #14. Octave 4.0.0-rc4 was started after replacing the file. The line in the printed figure still starts at (2,2), not (1,1) (see the attached fooDifferentLiboctinterp.eps).

You can count on me continuing to run tests on this issue in the future because it is really troublesome for my daily work.

(file #33897, file #33898)

Martin Kunz <mkunz>
Mon 04 May 2015 03:21:49 PM UTC, comment #18: 

@jwe: Has anyone tried running the version of Octave without Qt linked in?  The mere presence of Qt seems to upset things because of the multiple threads and we might be looking at another one of those issues.

The test would be to start Octave with --no-gui-libs, and then use the FLTK toolkit to print a figure and see if there are any problems.  It would still be using gl2ps and ghostscript so the important printing path would get exercised.

Rik <rik5>
Group administrator
Mon 04 May 2015 01:28:25 PM UTC, comment #17: 

I'm tagging this as a blocker and a regression as I'm told that even on systems where printing is failing now it used to work with 3.8.2.  It seems important to me that we have printing working correctly with the new release.

John W. Eaton <jwe>
Group administrator
Sat 02 May 2015 12:08:41 AM UTC, comment #16: 

Martin: Since you have two different systems that produce different results, can you check the version of OpenGL that is installed on the two systems?  The graphics hardware that they have?

If that's not easy to do, then I think I need to add something to Octave that will allow us to query and display that info.  That would probably be a good thing to do anyway...

John W. Eaton <jwe>
Group administrator
Sat 02 May 2015 12:02:07 AM UTC, comment #15: 

When I say it worked previously with 3.8.2, I mean that it worked on the same hardware and system with 3.8.2 where we see the failure with the current release candidate.  I'm attaching the changes I made to generate the modified liboctinterp-3.dll file that I posted.  These changes reverse nearly all of the changes to gl2ps-renderer.{h,cc} that have been made since 3.8.2.

(file #33867)

John W. Eaton <jwe>
Group administrator
Fri 01 May 2015 11:54:26 PM UTC, comment #14: 

I've done some more investigating and it seems that on systems where this problem happens gl2ps is getting a bad set of points from the OpenGL buffer.  Strange.  It also seems to affect printing with FLTK, so it doesn't seem likely be a threading issue.  And it worked previously in 3.8.2 with FLTK.

I'm still unable to duplicate the problem, but the person I've been working with might not be able to help more until next week.  If someone who is experiencing this problem could help me with debugging before then I would really appreciate it.

The first thing I need someone to try is to install ftp://alpha.gnu.org/gnu/octave/octave-4.0.0-rc4-installer.exe and then verify that there is a problem with printing the plot


plot (1:10)


(missing axes lines or missing segments from the line).

Next, exit Octave and replace the file bin/liboctinterp-3.dll with the following version: http://jweaton.org/liboctinterp-3.dll

Then try to print the same plot(1:10) graph again and see if the generated file looks correct and report the results here.

John W. Eaton <jwe>
Group administrator
Thu 30 Apr 2015 02:13:49 PM UTC, comment #13: 

Could you try replacing the file share\octave\4.0.0-rc4\m\plot\util\print.m with the attached version and then try


  plot (1:10)
  print ("foo.eps", "-depsc", "-debug")


and then attach the resulting foo.eps file here?

Be sure to restart Octave after replacing the print.m file so that the new version is used instead of any previous version that Octave has already parsed.

This time, you should see


  epstool command: ' cat > 'foo.eps' '
  opengl-pipeline: ' cat > foo.eps '


My goal here is to capture the output directly from gl2ps to see whether it is corrupted.  I think that should narrow the problem down to either gl2ps or ghostscript.

(file #33843)

John W. Eaton <jwe>
Group administrator
Thu 30 Apr 2015 11:45:17 AM UTC, comment #12: 

In the hope that it might help to find the cause for these problems: I do get other random errors on machine one when it comes to plotting. The first one is described in bug #44915, the other occurs more scarcely. Usually when I run a script that involves a lot of plotting, I get something like this:


[normal output from the script]
line at line 56 column 8
    __plt__>__plt2vv__ at line 500 column 10
    __plt__>__plt2__ at line 246 column 14
    __plt__ at line 113 column 17
    plot at line 220 column 10
    lplSimulatedVerticalProfile at line 328 column 3
error: octave_base_value::convert_to_str_internal (): wrong type argument '<unknown type>'
error: called from
    __line__ at line 117 column 21
    line at line 56 column 8
    __plt__>__plt2vv__ at line 500 column 10
    __plt__>__plt2__ at line 246 column 14
    __plt__ at line 113 column 17
    plot at line 220 column 10
    lplSimulatedVerticalProfile at line 328 column 3
error: octave_base_value::convert_to_str_internal (): wrong type argument '<unknown type>'
error: called from
    __line__ at line 117 column 21
    line at line 56 column 8
    __plt__>__plt2vv__ at line 500 column 10
    __plt__>__plt2__ at line 246 column 14
    __plt__ at line 113 column 17
    plot at line 220 column 10
    lplSimulatedVerticalProfile at line 328 column 3
error: octave_base_value::double_value (): wrong type argument '<unknown type>'
error: called from
    __line__ at line 117 column 21
    line at line 56 column 8
    __plt__>__plt2vv__ at line 500 column 10
    __plt__>__plt2__ at line 246 column 14
    __plt__ at line 113 column 17
    plot at line 220 column 10
    lplSimulatedVerticalProfile at line 328 column 3


Running the script again produces all the plots which I expect and no error.

Martin Kunz <mkunz>
Thu 30 Apr 2015 11:33:50 AM UTC, comment #11: 

I executed the commands you proposed in your last comment on the second machine (the one that produces good files) and got exactly the same results as on the problematic machine (checked with diff).

Martin Kunz <mkunz>
Thu 30 Apr 2015 09:33:49 AM UTC, comment #10: 

In a fresh instance of Octave (no print command executed since creation) I get:


>> plot (1:10)
>> print ('foo.eps', '-depsc', '-debug')
warning: print.m: epstool binary is not available.
Some output formats are not available.
warning: called from
    __print_parse_opts__ at line 382 column 9
    print at line 292 column 8
epstool command: 'C:\Octave\Octave-4.0.0-rc3\bin\gs.exe -dQUIET -dNOPAUSE -dBATCH -dSAFER -sDEVICE=eps2write -sOutputFile=""foo.eps"" -'
opengl-pipeline: 'C:\Octave\Octave-4.0.0-rc3\bin\gs.exe -dQUIET -dNOPAUSE -dBATCH -dSAFER -sDEVICE=eps2write -sOutputFile=""foo.eps"" -'
>> system ("gs -v")
GPL Ghostscript 9.16 (2015-03-30)
Copyright (C) 2015 Artifex Software, Inc.  All rights reserved.
ans = 0


On subsequent print commands there is no warning about a missing epstool binary.

This morning I also run my plot test script on a different machine under the same version of Windows (Windows 7 Enterprise 64bit Service Pack 1) and the same version of Octave (4.0.0-rc3). All the figures had complete axes and the curves where fine. The only flaw I spotted were vertically shifted x labels, but this occured only in the SVG files.

Martin Kunz <mkunz>
Thu 30 Apr 2015 06:03:36 AM UTC, comment #9: 

What does the following show for you in the command window?

+verbatin+
  plot (1:10)
  print ('foo.eps', '-depsc', '-debug')
-verbatim-

I think it should print something like


  epstool command: 'C:\Users\jwe\Desktop\octave-2015-04-29-15-58\bin\gs.exe -dQUIET -dNOPAUSE -dBATCH -dSAFER -sDEVICE=eps2write -sOutputFile=""foo.eps"" -'
  opengl-pipeline: 'C:\Users\jwe\Desktop\octave-2015-04-29-15-58\bin\gs.exe -dQUIET -dNOPAUSE -dBATCH -dSAFER -sDEVICE=eps2write -sOutputFile=""foo.eps"" -'


in the Octave command window.  Also, what does


  system ("gs -v")


return for you?

John W. Eaton <jwe>
Group administrator
Mon 27 Apr 2015 03:42:44 PM UTC, comment #8: 

I see the following differences in output from my test and the files you produced, plus differences in the compressed data that describes the EPS figure.  I have no clue why this is happening as we are using the same version of ghostsript.  This seems to point to a bug in ghostscript, or that we are misconfiguring or miscompiling it somehow.


--- good/q1.eps        2015-04-27 10:15:11.278533000 -0500
+++ bad/f1.eps        2015-04-27 10:27:09.888122675 -0500
@@ -1,9 +1,9 @@
 %!PS-Adobe-3.0 EPSF-3.0
-%%BoundingBox: 39 29 523 405
-%%HiResBoundingBox: 39.30 29.20 522.40 405.00
+%%BoundingBox: 29 7 544 406
+%%HiResBoundingBox: 29.50 7.30 543.10 405.90
 %%Creator: GPL Ghostscript 916 (eps2write)
 %%LanguageLevel: 2
-%%CreationDate: D:20150427081510-07'00'
+%%CreationDate: D:20150427114106+02'00'
 %%Pages: 1
 %%EndComments
 %%BeginProlog
@@ -1270,109 +1270,117 @@
 endobj
 %%EndPageSetup
 5 0 obj
-<</Filter/LZWDecode/Length 23381>>stream
+<</Filter/LZWDecode/Length 24287>>stream


John W. Eaton <jwe>
Group administrator
Mon 27 Apr 2015 03:04:28 PM UTC, comment #7: 

I still haven't been able to duplicate the problem after multiple runs of the plotTest script.  Hmm..

John W. Eaton <jwe>
Group administrator
Mon 27 Apr 2015 02:45:02 PM UTC, comment #6: 

OK, so it seems to me that if the EPS files are incorrect, then the problem is in gl2ps or some other part of the process of generating the PS file.  I'm not sure how to debug this problem.

I haven't been able to reproduce the problem, even on a Windows system.  But you say that the behavior is somewhat random, so I suppose that it is possible that I simply need to run the script multiple times to show the problem.

John W. Eaton <jwe>
Group administrator
Mon 27 Apr 2015 09:42:29 AM UTC, comment #5: 

Of course! I added eps output to my script with:


print -depsc -r0 f1.eps


You find f(1|2|3).eps attached. Note that the behaviour of print() is not deterministic, first because of the irregular occuring error described in bug #44915 and second because the appearance of the figures changes when I run the script several times in a row. Figure 3 had the strange extra lines in the PDF uploaded earlier, now I saw the curve with a broken line (but no extra lines) or even a solid line (the bottom axis, however, is still missing). PDF, PNG and EPS look the same.

Martin Kunz <mkunz>
Sun 26 Apr 2015 04:10:13 PM UTC, comment #4: 

Since printing seems to work properly for me on my Debian system, I don't think this is a problem with conversion from PS to other formats.  But it would help to see whether the PS files that are generated are correct.  Then we swould know whether the problem was with gl2ps or the conversion from PS to the other formats (ghostscript?).  In any case, this seems to me to be a Windows-specific problem.

John W. Eaton <jwe>
Group administrator
Sat 25 Apr 2015 03:55:54 AM UTC, comment #3: 

Could you please generate EPS equivalents of the figures and attach?  I.e., print -depsc.

One of the concerns I've had about the approach of creating all output formats as derivative of the EPS file is aliasing effects and other unexpected behavior with the ps2xyz translation.

Dan Sebald <sebald>
Fri 24 Apr 2015 12:35:34 PM UTC, comment #2: 

Yes, but they are a bit different. I attach f1.pdf, f2.pdf and f3.pdf, all created from FLTK figures.

  • In figure 1 the bottom axis is missing.
  • In figure 2 the bottom axis is missing and the curve has small gaps.
  • Figure 3 looks pretty much like the version drawn by Qt.


(file #33768, file #33769, file #33770)

Martin Kunz <mkunz>
Fri 24 Apr 2015 11:17:01 AM UTC, comment #1: 

Hi,

Thanks for your bug report.
The Greek letters not being printed correctly is a known issue (see bug #42320). The fact that it works in svg format when using utf8 characters is because svg unlike eps (from which pdf and png are converted) does handle utf8 natively if the viewer does.

Does FLTK also produce missing axes and corrupted lines?

Pantxo Diribarne <pantxo>
Group Member
Fri 24 Apr 2015 10:51:58 AM UTC, original submission:  

I am using Octave 4.0.0-rc3 from ftp://alpha.gnu.org/gnu/octave/ under Windows 7 64bit.

I love the new Qt graphics toolkit because it is fast and the onscreen figures look nice. However, printing Qt figures to PNG, PDF and SVG files produces strange results.

The attached script plotTestQt.m creates 3 rather simple figures and saves them to PNG, PDF and SVG. The resulting files are attached, as well as screenshots of the figure windows. The latter look all fine, even the Omega is printed correctly (With the exception of figure 3, where the Unicode char for Omega is used in the ylabel command).
However, all of the files have issues:

  • Figure 1 has no bottom axis. The Greek Omega appears as "Omega" in the PDF and PNG file and as "Omega" in the SVG.
  • Figure 2 consists only of axes labels and tick labels in all three formats. Axes, tick marks and curves are completely missing. The Greek Omega appears as "Omega" in the PDF and PNG file and as "Omega" in the SVG.
  • Figure 3 has nice axes and labels, but the curve is totally screwed: It is intersected many times and has a lot of straight lines connecting the first point to points on the curve. The Greek Omega is a strange symbol in the PDF and PNG file and displayed correctly (!) in the SVG file.



Martin Kunz <mkunz>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #37929:  opengl.patch added by lostbard (5KiB - text/x-patch)
file #33945:  Screenshot-com37dat.png added by sebald (17KiB - image/png)
file #33932:  gl2ps-show-vertex-diffs.txt added by jwe (4KiB - text/plain)
file #33925:  f2greenBG.eps added by mkunz (99KiB - application/postscript)
file #33897:  f3_octave3.8.0.eps added by mkunz (114KiB - application/postscript)
file #33898:  fooDifferentLiboctinterp.eps added by mkunz (90KiB - application/postscript)
file #33867:  diffs.txt added by jwe (10KiB - text/plain)
file #33843:  print.m added by jwe (24KiB - text/x-objcsrc)
file #33807:  f2.eps added by mkunz (99KiB - application/postscript)
file #33808:  f3.eps added by mkunz (137KiB - application/postscript)
file #33806:  f1.eps added by mkunz (113KiB - application/postscript)
file #33768:  f1.pdf added by mkunz (24KiB - application/pdf)
file #33769:  f2.pdf added by mkunz (11KiB - application/pdf)
file #33770:  f3.pdf added by mkunz (66KiB - application/pdf)
file #33765:  q1Screenshot.png added by mkunz (24KiB - image/png)
file #33766:  q2Screenshot.png added by mkunz (28KiB - image/png)
file #33767:  q3Screenshot.png added by mkunz (29KiB - image/png)
file #33764:  q3.svg added by mkunz (177KiB - image/svg+xml)
file #33760:  q2.png added by mkunz (5KiB - image/png)
file #33761:  q2.svg added by mkunz (3KiB - image/svg+xml)
file #33762:  q3.pdf added by mkunz (67KiB - application/pdf)
file #33763:  q3.png added by mkunz (58KiB - image/png)
file #33756:  q1.pdf added by mkunz (24KiB - application/pdf)
file #33757:  q1.png added by mkunz (16KiB - image/png)
file #33758:  q1.svg added by mkunz (58KiB - image/svg+xml)
file #33759:  q2.pdf added by mkunz (3KiB - application/pdf)
file #33755:  plotTestQt.m added by mkunz (798B - text/plain)

 

Depends on the following items: None found

Digest:
   bug dependencies.

 

Carbon-Copy List
  • -email is unavailable- added by johasixt
  • -email is unavailable- added by mmuetzel (Posted a comment)
  • -email is unavailable- added by dasergatskov (Posted a comment)
  • -email is unavailable- added by lachlan (Posted a comment)
  • -email is unavailable- added by lostbard (Posted a comment)
  • -email is unavailable- added by godfrey (Posted a comment)
  • -email is unavailable- added by rik5 (Posted a comment)
  • -email is unavailable- added by jwe (Posted a comment)
  • -email is unavailable- added by sebald (Posted a comment)
  • -email is unavailable- added by pantxo (Posted a comment)
  • -email is unavailable- added by mkunz (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 25 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2018-09-03 johasixt Carbon-Copy- Added johasixt
    2018-07-03 rik5 Dependencies- bugs #53845 is dependent
    2016-08-08 rik5 Severity5 - Blocker 3 - Normal
        Priority7 - High 5 - Normal
        Item GroupRegression Performance
        Release4.0.0 dev
        SummaryPrinting Qt-drawn figures produces faulty files --softwareopengl option for performance
    2016-07-17 lostbard Attached File- Added opengl.patch, #37929
    2015-06-02 jwe Release4.0.0-rc3 4.0.0
    2015-05-06 sebald Attached File- Added Screenshot-com37dat.png, #33945
    2015-05-06 jwe Attached File- Added gl2ps-show-vertex-diffs.txt, #33932
    2015-05-05 mkunz Attached File- Added f2greenBG.eps, #33925
    2015-05-04 mkunz Attached File- Added f3_octave3.8.0.eps, #33897
        Attached File- Added fooDifferentLiboctinterp.eps, #33898
    2015-05-04 jwe Severity3 - Normal 5 - Blocker
        Priority5 - Normal 7 - High
        Item GroupIncorrect Result Regression
    2015-05-02 jwe Attached File- Added diffs.txt, #33867
    2015-04-30 jwe Attached File- Added print.m, #33843
    2015-04-27 mkunz Attached File- Added f1.eps, #33806
        Attached File- Added f2.eps, #33807
        Attached File- Added f3.eps, #33808
    2015-04-24 mkunz Attached File- Added f1.pdf, #33768
        Attached File- Added f2.pdf, #33769
        Attached File- Added f3.pdf, #33770

    Back to the top

    Powered by Savane 3.13-f8d8.
    Corresponding source code