bugGNU Octave - Bugs: bug #53167, format long doesn't adjust...

 
 

bug #53167: format long doesn't adjust precision to 7 digits when displaying single() values

Submitter:  Rik <rik5>
Submitted:  Fri 16 Feb 2018 01:09:07 AM UTC
   
 
Category:  Interpreter Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Matlab Compatibility
Status:  Fixed Assigned to:  None
Originator Name:  Open/Closed:  * Closed
Release:  * dev Operating System:  * Any
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Fri 23 Feb 2018 04:34:29 AM UTC, comment #14: 

Fixed and verified.  Closing report.

Here is an example of how the new behavior clearly helps the programmer immediately identify single values in a container data structure.


format long
x = {[single(pi)] ; [pi]}
x =
{
  [1,1] =  3.1415927
  [2,1] =  3.141592653589793
}



Rik <rik5>
Group administrator
Wed 21 Feb 2018 06:06:12 PM UTC, comment #13: 

You'll see that I removed output_max_field_width.  I did this because I couldn't see the point of having it and had forgotten why it was needed.  But now I see that bad things happen without it, for example, the following


fixed_point_format (false);
format short
for x = 1:3:12
  randn (10) * 10^x
end


previously would be displayed in E format once the values all became too large for the max field width.

I suppose I should restore this behavior, but I'm not sure whether there should be separate limits for single and double values.

John W. Eaton <jwe>
Group administrator
Wed 21 Feb 2018 05:30:46 PM UTC, comment #12: 

I pushed a series of changesets ending with

  http://hg.savannah.gnu.org/hgweb/octave/rev/4d945f2e5914

These changes should improve compatibility.  I may do a little more work with templates here to remove some more duplicate code.  But I think this is reasonably good for now.

John W. Eaton <jwe>
Group administrator
Tue 20 Feb 2018 04:26:57 AM UTC, comment #11: 

It feels obfuscating to use either digits10 + 1 or max_digits10 - 1 when we already know exactly the precision we need to remain Matlab compatible.  I would just put it in a const expression with a note about why we use this value.

Rik <rik5>
Group administrator
Tue 20 Feb 2018 03:00:39 AM UTC, comment #10: 

There is also max_digits10, but that is 17 for double and 9 for float.

John W. Eaton <jwe>
Group administrator
Tue 20 Feb 2018 02:21:14 AM UTC, comment #9: 

It's doing the right thing. But we might want to use digits10 + 1 for floating point types. The value of digits10 isn't the maximum number of digits that can be represented at all, it's the maximum number of digits that can always be represented without any loss of precision.

flintmax("double") is close to 1e16, but not quite. and flintmax("single") is closer to 1e7. So 15 and 6 are correct.

http://en.cppreference.com/w/cpp/types/numeric_limits/digits10

Mike Miller <mtmiller>
Group Member
Tue 20 Feb 2018 12:46:16 AM UTC, comment #8: 

I don't think digits10 does the right thing.  I tried


   cout << "Max sig. digits double: " << numeric_limits<double>::digits10 << endl;
   cout << "Max sig. digits single: " << numeric_limits<float>::digits10 << endl;


and get


Max sig. digits double: 15
Max sig. digits single: 6



Rik <rik5>
Group administrator
Tue 20 Feb 2018 12:39:14 AM UTC, comment #7: 

Another reason for the output_precision variable was to allow more fine-grained control than just format long or short.  Is it acceptable to drop that feature?  I'm OK with limiting output to no more than the number of digits that are meaningful, but it would be nice to maintain the ability to get, say 10 digits for doubles, not just 8 or 16.

That's why I asked whether there should be just one value of output_precision, and then limit the actual output to the max number of meaningful digits for the type.  If we do that and have only one output_precision value, then users could set output_precision to whatever value and we would impose a limit when printing.  But if we have separate output_precision values for single and double, we can limit their ranges at the time they are set from a user-provided value.

John W. Eaton <jwe>
Group administrator
Tue 20 Feb 2018 12:33:49 AM UTC, comment #6: 

std::numeric_limits<T>::digits10 doesn't change based on value.  It's a constant for the type T.  I referred to it because I think it's the number of digits we are looking for.

John W. Eaton <jwe>
Group administrator
Tue 20 Feb 2018 12:31:21 AM UTC, comment #5: 

The following is essentially what Octave does now when you set output_precision to 75:


#include <cmath>
#include <iomanip>
#include <iostream>

int
main (void)
{
  std::cout.flags (std::ios::showpoint);
  std::cout << std::setprecision (70) << std::setw (75) << M_PI << std::endl;
  return 0;
}


On my system, this prints:


     3.141592653589793115997963468544185161590576171875000000000000000000000


I believe this is the same as


#include <math.h>
#include <stdio.h>

int
main (void)
{
  printf ("%80.75f\n", M_PI);
  return 0;
}


Maybe it's stupid, but when these format functions were originally written for Octave, I think the aim was to allow this kind of formatting.

John W. Eaton <jwe>
Group administrator
Mon 19 Feb 2018 10:16:02 PM UTC, comment #4: 


> Currently, a side effect of "format long" and "format short" is to set output_precision. But that's not specific to single or double values. So we have a choice or two to make.
>
> Should we introduce a separate output_precision variable for single precision values and have format long/short set both of those values?


That would be one way to handle it.

>
> If not, then should we limit the display of values to the maximum number of digits that double and single values can represent (std::numeric_limits<T>::digits10 in C++)?


Is this really worth it?  The values aren't going to change given that we use IEEE standards for single and double, so why not specify the value we want?  I'm concerned that for double there are 15-17 significant figures.  Would std::numeric_limits<T>::digits10 change based on the actual value?  That would be weird.  We want a fixed value and it should probably be 16.  Given that we know what we want it doesn't seem like we need to query std::numeric_limits.

>
> If we have a limit, should that limit apply to both single and double precision values, and should it be imposed on the value of the output_precision variable(s) or just as a limit inside the print functions?


???  I don't think I understand.  Doubles should be displayed with 16 significant figures and singles with 8 so there should be different limits.

>
> Should we eliminate the output_precision and format_max_field_width variables and just use the format function to set long/short/whatever format styles?


This makes more sense since it is the value itself, in combination with the format, which determines how it is displayed.  For example, floating point numbers which are true integers ignore the output_precision argument.  It seems reasonable that the underlying value, be it double or single, determines how many digits are displayed rather than just a global variable.

>
> None of these changes would affect *printf, so you could always do something like "fprintf ('%.200f\n', val)" if you want to see the digits that the floating point conversion routine generates when you ask for more precision than is actually available.


There is a bug report somewhere about one of the conversion routines generating extra nonsense digits after the first 16.  Apparently Matlab doesn't do this.  For example,


sprintf ('%30.30d\n', 1e26)
ans =    100000000000000004764729344


This should just be a '1' with 26 '0's after it.

Rik <rik5>
Group administrator
Sun 18 Feb 2018 05:47:35 PM UTC, comment #3: 

Currently, a side effect of "format long" and "format short" is to set output_precision.  But that's not specific to single or double values.  So we have a choice or two to make.

Should we introduce a separate output_precision variable for single precision values and have format long/short set both of those values?

If not, then should we limit the display of values to the maximum number of digits that double and single values can represent (std::numeric_limits<T>::digits10 in C++)?

If we have a limit, should that limit apply to both single and double precision values, and should it be imposed on the value of  the output_precision variable(s) or just as a limit inside the print functions?

Should we eliminate the output_precision and format_max_field_width variables and just use the format function to set long/short/whatever format styles?

None of these changes would affect *printf, so you could always do something like "fprintf ('%.200f\n', val)" if you want to see the digits that the floating point conversion routine generates when you ask for more precision than is actually available.

John W. Eaton <jwe>
Group administrator
Fri 16 Feb 2018 05:11:58 PM UTC, comment #2: 

Seems like a good idea.  I think we should arrange for 8 significant digits to be displayed, which in fixed point format is 1 digit + '.' + 7 digits which would be equivalent to Matlab.

Rik <rik5>
Group administrator
Fri 16 Feb 2018 08:06:45 AM UTC, comment #1: 

Given the way the display code is structured in pr-output.cc, I think the best solution is to define separate display functions for double and float values.

I see the following changset as a first step in that direction:

  http://hg.savannah.gnu.org/hgweb/octave/rev/39186eac5a05

My plan is to use templates to define separate display functions for real and float values without having to duplicate much code.  Currently, float is transformed to double and the same function is used for both types.  Once this change is made, it should be fairly simple to ensure that format long doesn't print more decimal digits than are actually available.

John W. Eaton <jwe>
Group administrator
Fri 16 Feb 2018 01:09:07 AM UTC, original submission:  

According to documentation, Matlab only displays 7 digits after the decimal point when showing class single values.  This is useful because it reinforces to the user that digits beyond the 7th are not meaningful.

In Octave, a long format always displays 15 digits after the decimal point.

Octave:


octave:1> format long
octave:2> single (pi)
ans =  3.14159274101257



Rik <rik5>
Group administrator

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by mtmiller (Posted a comment)
  • -email is unavailable- added by jwe (Posted a comment)
  • -email is unavailable- added by rik5 (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 4 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2018-02-23 rik5 StatusReady For Test Fixed
        Open/ClosedOpen Closed
    2018-02-21 jwe StatusIn Progress Ready For Test
    2018-02-16 rik5 StatusNone In Progress

    Back to the top

    Powered by Savane 3.13-d3ae.
    Corresponding source code