bugGNU Octave - Bugs: bug #32980, OpenGL plotting backend uses...


bug #32980: OpenGL plotting backend uses single precision, not double

Submitted by:  Michael Godfrey <godfrey>
Submitted on:  Sun 03 Apr 2011 01:04:16 AM UTC  
Category: Plotting with OpenGLSeverity: 3 - Normal
Priority: 5 - NormalItem Group: Incorrect Result
Status: NoneAssigned to: None
Originator Name: Michael GodfreyOpen/Closed: Open
Release: devOperating System: GNU/Linux

Add a New Comment(Rich Markup)

You are not logged in

Please log in, so followups can be emailed to you.


(Jump to the original submission Jump to the original submission)

Sun 20 Nov 2016 11:52:56 AM UTC, comment #24:

As discussed below this is a feature of OpenGL. It is 32bit
and there is no plan to change this even though some graphics
hardware supports 64bit. And, there is no real benefit from
use of 64bit

So, as again discussed below, the change that is required is
for the data to be tested for within 32bit range and rescaled
if not. (As is done in Matlab.)

Michael Godfrey <godfrey>
Project Member
Sat 19 Nov 2016 01:56:31 PM UTC, comment #23:

This behavior is still present in Octave 4.2.0.

Hartmut <hardy>
Fri 24 Feb 2012 09:20:13 PM UTC, comment #22:

Yes, you're right!

Now that given a bit of thought, log10(double) can reliably represented as a single.

Ben Abbott <bpabbott>
Project Member
Fri 24 Feb 2012 09:08:41 PM UTC, comment #21:

Aren't log scales already handled? For example, the following already works properly for me:

John W. Eaton <jwe>
Project Administrator
Fri 24 Feb 2012 08:38:34 PM UTC, comment #20:

Yes. I think we'll need to avoid m-files to avoid triggering listeners.

My fist inclination was that we'd place everything in _init_fltk_.cc, but there's probably a nifty OO way of doing this ? (I'm a procedural guy).

We'll also need to consider log scale plots. They'll be a bit more tricky, but the same approach will work.

Ben Abbott <bpabbott>
Project Member
Fri 24 Feb 2012 08:14:48 PM UTC, comment #19:

Mapping the data seems fine. To get it to work I had to move the mapping of {x,y,z}tick ahead of the for loop because the call to set (h, 'ydata', ydata) updates the ylimits which are then wrong in the ymap function.

How were you thinking of implementing this? I just reread the discussion and JWE seemed reluctant to put this in axes.m. You mentioned putting equivalent code in _init_fltk_.cc?

Rik <rik5>
Project Administrator
Fri 24 Feb 2012 05:54:14 PM UTC, comment #18:

Any problem with mapping the {x,y,z}data ?

Ben Abbott <bpabbott>
Project Member
Fri 24 Feb 2012 05:15:29 PM UTC, comment #17:

The approach is right but it is slightly more complicated. The data needs to be shifted AND normalized. For example, try this simple plot.

This produces a blank plot for me because 1e40 is beyond what is representable in a variable of class single. Offsetting in this case doesn't help because even after offsetting the new y-axis range is from -5e40 to +5e40 and both of those values are still not representable.

I think you can get this to work by both shifting by the mean and normalizing by the mean as well. I just did some command line work and dividing ydata by 1e40, ytick by 1e40, and ylim by 1e40 caused the correct graph to show up.

Rik <rik5>
Project Administrator
Fri 24 Feb 2012 03:45:20 PM UTC, comment #16:

It should be possible to shift (and normalized?) the xdata, ydata, zdata, cdata in _init_fltk_.cc.

A simple example from the command line that illustrates the problem for xdata is below.

By offsetting the xdata by its median the problem by be resolved. After running the commands above, add those below.

Is this a proper approach ?

Ben Abbott <bpabbott>
Project Member
Sat 18 Feb 2012 05:35:11 AM UTC, comment #15:

I suspect that Michael Goffioul knows how to do this.
It would be a good idea to test all data being processed
for plotting to ensure that the plot comes out right.
For OpenGL this means doing a transform that maps all the
data to within reals. gl-render.cc is a candidate place
to do this. But it would also be useful to
check for very small data value differences so that the
plot scales and tic values can be made meaningful.
Now, an attempt to plot: plot(1:0.000001:0.000009)
does not work well, due to axis values being truncated.
Note that Matlab does not do this right either.

Michael Godfrey <godfrey>
Project Member
Sat 18 Feb 2012 01:26:20 AM UTC, comment #14:

ping Any new thoughts/progress here?

Ben Abbott <bpabbott>
Project Member
Mon 25 Apr 2011 03:30:53 PM UTC, comment #13:


The code is specific in the sense that it tests for
the one current backend that restricts plot data to
single. I looked in the fltk code, but could not find
a way to implement these tests there. Also, it is
possible that there will be other backends that only
support single precision.

Currently the code uses realmax("double") for all cases
other than fltk, but this is pretty meaningless. This
test can never, currently, fail. I will change the code
to be fltk/OpenGL specific.

I thought about the rescaling and this should be possible,
but a lot more work in code that I am not familiar with.
It would be great if someone would do that.

In the meantime, it is not good that users may experience
mysteriously incorrect plots due to overflow of singles
in fltk/OpenGL.

Michael Godfrey <godfrey>
Project Member
Mon 25 Apr 2011 02:52:26 PM UTC, comment #12:

The code in axes.m should be generic and not have special cases for specific toolkits. So it seems to me that it would be better to fix this in the part of Octave that works directly with the fltk/opengl toolkit. Since we have complete control over the axes labels in that code, it seems that all we would need to do is detect the condition there, scale the values to plot and then place axis labels appropriately. That should allow us to handle the full range of double precision values even if OpenGL doesn't handle that internally.

John W. Eaton <jwe>
Project Administrator
Sun 24 Apr 2011 12:27:31 AM UTC, comment #11:

Here is another update. I found that all the tests
can be (probably must be) made in axes.m. So, this
fix only updates axes.m

It can be implemented in 2 forms, using a variable
do_reset. If do_reset is non-zero, the code changes
the axis limits be, possibly, "in range." Here
"in range" means that the plot will have correct
axes and display any data that fits in the axes.

It is likely that the code should be used with do_reset = 0.
Then the axes are not changed, but a warning is issued.
The resulting plot will be garbage.

I plan to do more testing, but this is much better than
the last try.

Attached are 2 files:
diff -c on axes.m and the axes.m file. For testing
just place the axes.m file in the current directory.


(file #23293, file #23294)

Michael Godfrey <godfrey>
Project Member
Fri 22 Apr 2011 04:46:26 PM UTC, comment #10:

Thanks Rik.

Since choice 1 would be at best unreliable I agree that
choice 2 is better, and easier. It may be appropriate
to issue the warning at a value less than realmax("single")
and change the message to read something like "axis
and data values near realmax("single") or realmin("single")
may produce incorrect plots due to overflow/underflow"

And, your elimination of for loops is appropriate, of
course. I try to do that too, but this code was a quick
test and copied from similar loops in axis.m...
Rounding to zero may be right, too.


Michael Godfrey <godfrey>
Project Member
Fri 22 Apr 2011 04:21:32 PM UTC, comment #9:

My inclination is for option 2. I subscribe to the philosophy that the user has more information, and is more intelligent, than the program and can make a better choice on how to resolve an error situation. Thus, I think we can just issue a warning and produce an unreadable mess. The user may then choose to scale their graph, perhaps choose the gnuplot toolkit, change the plot type to semilogy, etc. What we have right now isn't workable because one can reach an error state, such as having one of the limits set to Inf, that is impossible to escape.

Also, regarding your patch. I think it would be acceptable, when the value is below realmin, to round down to 0 rather than round up to realmin.

Finally, even though the for loop in the patch is over only a few items, I hate to see them in Octave. Here is a sample way to avoid the loops by using indexing that you might consider or modify.

Rik <rik5>
Project Administrator
Fri 22 Apr 2011 06:43:05 AM UTC, comment #8:


The problem with sombrero was that the test in axes.m
was incorrect. I now have a patch which works for

However, this has revealed a new problem. The limit
for "single" in the case of the sombrero plot is
approximately 4e+35. This is a lot smaller than for
the 2D plots where the limit is about 1.3e38. Somewhere
in the code values must occur which are above
realmax("single") even though the peak of the sombrero
is only about 4e35.

So, there are at least 2 choices:

1. Find max and min values for the axes which will
allow a "correct" plot in all known cases and
reset the axes to these values. This will produce
a valid plot of all the data that fix within the
reset axes. A max/min value around +-e35 may work.

2. Just issue the warning and let the plot appear as
an unreadable mess. If this uses the exact
realmax/min("single") there will be error cases that
are not caught.

While waiting for comments, I will clean up the code.


Michael Godfrey <godfrey>
Project Member
Fri 22 Apr 2011 01:38:55 AM UTC, comment #7:


Thanks for trying the patch. I am not surprised
that it failed the sombrero. I had only tried 2D
plots mainly to try to check that the code was in the
right places. It does check all three dimensions, but
something is wrong for 3D.

If you agree that this code is capable of doing the
right tests, I will improve it (also making the now
duplicated code into a function).


Michael Godfrey <godfrey>
Project Member
Fri 22 Apr 2011 01:11:26 AM UTC, comment #6:

Unfortunately the patch does not seem to operate correctly on some plots.

Try the following example:

Without the patch, it produces the sombrero figure. With the patch it produces a tiny fraction of a surface within the axis limits [0 1 0 1 0 1]. I can get it to display the whole figure with axis("auto"), but it indicates that something isn't right.

Rik <rik5>
Project Administrator
Wed 20 Apr 2011 11:36:10 PM UTC, comment #5:

Here is code that appears to address the scale
problem in fltk due to use of single.

It was claimed that the code could be placed in
axis.m (around line 280) instead of in axes.m
but after putting the code in axis.m I found
that essentially the same code was needed in axes.m.
For example plot([1 10e40]) does not call axis.m
at all. So, for now, there is duplicated code.
This code should be made into a called function.

I am posting this code for review. There may well be
a better way to do this. The code issues a warning and
rescales the plot. This will prevent various forms
of inappropriate behavior.

After any comments I will try the cleanup.


(file #23281)

Michael Godfrey <godfrey>
Project Member
Tue 05 Apr 2011 06:30:15 PM UTC, comment #4:

It appears that a place where the axes can be
checked for validity is after line 47 in axes.m.

Two questions for anyone (John?) who knows this
1. Is this the right place?

2. Is it OK to put code which is conditional on
the current backend at this point?

I notice that currently the command:
axis([0 Inf 0 200]) is processed without
complaint, but it obviously does not work and
trying to recover by axis([0 200 0 200]) which
is what the axes were before does not bring back
the plot correctly.

So, it seems a good idea to provide an axes validity
check in any case. Currently the checks for gnuplot
(essentially valid range -max(double) to max(double))
and fltk/OpenGL are different, but both are needed.

Michael Godfrey <godfrey>
Project Member
Tue 05 Apr 2011 03:24:33 AM UTC, comment #3:

I added a Caution note to the documentation about only single precision being available with the FLTK toolkit. (http://hg.savannah.gnu.org/hgweb/octave/rev/562e40e3d5db)

Rik <rik5>
Project Administrator
Tue 05 Apr 2011 02:44:51 AM UTC, comment #2:

I checked that Matlab does not have this
problem. But, I also noticed that Matlab
makes use of opengl.

So, there may be a way to get opengl to
work correctly.


Michael Godfrey <godfrey>
Project Member
Sun 03 Apr 2011 05:39:03 AM UTC, comment #1:

As Shai pointed out, the problem seems to be
in OpenGL which accepts double, but converts
to single. This is not really made clear in
the web pages I have read, but it is mentioned that
they the work in the precision available in the
GPU hardware which is typically 32bit. Newer
hardware provides 64bit, so there is some
indication that a future OpenGL will provide
64bit. In the meantime, this report could be
dealt with by providing a warning that the plot
cannot be rendered correctly due to data out of
range of representable values.

The current behavior is not good since it could
lead to plots which are seriously, but not always
obviously, in error.


Michael Godfrey <godfrey>
Project Member
Sun 03 Apr 2011 01:04:16 AM UTC, original submission:

A simple example is:

This problem does not exist for gnuplot, so the error
must be in fltk-dependent code.

Michael Godfrey <godfrey>
Project Member


(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):

Attached Files
file #23293:  axes.m added by godfrey (4KiB - application/vnd.wolfram.mathematica.package)
file #23294:  axes_2.diff added by godfrey (3KiB - application/octet-stream)
file #23281:  fltk_axes.diff added by godfrey (4KiB - application/octet-stream)


Depends on the following items: None found

   bug dependencies.


Carbon-Copy List
  • -unavailable- added by hardy (Posted a comment)
  • -unavailable- added by jordigh (Updated the item)
  • -unavailable- added by bpabbott (Posted a comment)
  • -unavailable- added by jwe (Posted a comment)
  • -unavailable- added by rik5 (Posted a comment)
  • -unavailable- added by godfrey (Submitted the item)

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only project members can vote.


    Please enter the title of George Orwell's famous dystopian book (it's a date):



    Follow 15 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Tue 26 Apr 2016 06:53:19 PM UTCmtmillerDependencies-=>bugs #47788 is dependent
    Wed 25 Nov 2015 04:44:54 PM UTCmtmillerDependencies-=>bugs #46529 is dependent
    Mon 31 Aug 2015 08:19:58 PM UTCmtmillerDependencies-=>bugs #45851 is dependent
    Thu 16 Jul 2015 10:50:37 AM UTCpantxoDependencies-=>bugs #45541 is dependent
    Sun 07 Jun 2015 03:50:22 PM UTCmtmillerDependencies-=>bugs #45271 is dependent
    Thu 30 Apr 2015 12:15:12 AM UTCmtmillerDependencies-=>bugs #44963 is dependent
    Fri 26 Dec 2014 03:29:02 PM UTCrik5Dependencies-=>bugs #33748 is dependent
    Thu 14 Mar 2013 02:38:16 PM UTCjordighSummaryfor backend FLTK, real is used for axes. double should be used=>OpenGL plotting backend uses single precision, not double
    Thu 14 Mar 2013 02:37:41 PM UTCjordighDependencies-=>bugs #38515 is dependent
    Thu 10 Jan 2013 01:57:10 PM UTCjordighDependencies-=>bugs #38054 is dependent
    Tue 16 Oct 2012 11:23:25 PM UTCrik5Dependencies-=>bugs #37553 is dependent
    Sun 24 Apr 2011 12:27:31 AM UTCgodfreyAttached File-=>Added axes.m, #23293
      Attached File-=>Added axes_2.diff, #23294
    Wed 20 Apr 2011 11:36:10 PM UTCgodfreyAttached File-=>Added fltk_axes.diff, #23281
    Mon 04 Apr 2011 05:30:23 PM UTCrik5Dependencies-=>bugs #32682 is dependent

    Back to the top

    Powered by Savane 3.1-cleanup1