bugGNU Octave - Bugs: bug #41359, Output formatting inconsistency...

 
 

bug #41359: Output formatting inconsistency for integers > 2^52

Submitted by:  None
Submitted on:  Mon 27 Jan 2014 02:40:44 AM UTC  
 
Category: InterpreterSeverity: 3 - Normal
Priority: 5 - NormalItem Group: Incorrect Result
Status: ConfirmedAssigned to: None
Originator Name: G. GoldenOriginator Email: -unavailable-
Open/Closed: OpenRelease: dev
Operating System: GNU/Linux

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

Thu 15 Dec 2016 11:09:12 AM UTC, comment #5:

This issue is still present in Octave 4.2.0.

Hartmut <hardy>
Mon 27 Jan 2014 04:45:42 PM UTC, comment #4:

I haven't looked into how libc handles this with printf, but sure it could be worth looking into.

And as to the savannah markup, don't worry about it, we are used to it, we've all messed up the markup from time to time. Reposting to fix markup mistakes is fine.

Mike Miller <mtmiller>
Project Administrator
Mon 27 Jan 2014 02:57:22 PM UTC, comment #3:

verbose -> verbatim on previous comment, sry.

Which begs the question: Is there a preview feature for submitting comments? Didn't see anything in a brief look thru the Savannah help doc, but perhaps just missed it...

Glenn Golden <grepfor>
Mon 27 Jan 2014 02:44:12 PM UTC, comment #2:

Interesting, I see what you mean. Need an integer test that's independent of the float EPS.

Fwiw -- perhaps nothing, just an observation -- printf %.16g does get it right:

+verbose+

> a = 2^52 + (-5:5);
> printf("a: %.16g\n", a)

a: 4503599627370491
a: 4503599627370492
a: 4503599627370493
a: 4503599627370494
a: 4503599627370495
a: 4503599627370496
a: 4503599627370497
a: 4503599627370498
a: 4503599627370499
a: 4503599627370500
a: 4503599627370501
-verbose-

So might be worth a look at how the integer test is being handled there.

Glenn Golden <grepfor>
Mon 27 Jan 2014 03:15:57 AM UTC, comment #1:

Yes, you are right that the values should be representable in 16 decimal digits. The basis of the decision whether a number can be displayed as an integer or not is failing for these cases. The output formatter then believes that the value is not strictly an integer and displays what you are seeing.

Here are essentially the computations that the formatter is doing to determine whether the value is an integer or not:

Since we've hit the limit of the significand, adding 0.5 rolls over into the next bit position, floor gives the wrong result, and it appears that a+1 is not an integer according to this formula.

Mike Miller <mtmiller>
Project Administrator
Mon 27 Jan 2014 02:40:44 AM UTC, original submission:

In "format long" display mode, with output_precision set to 16, odd integers between 2^52 and 2^53 -- all of which are exactly representable in IEEE-754 double precision, and have 16 or fewer decimal digits -- are nevertheless displayed in E format.

Example:

Note that the displayed values for the odd integers above 2^52 are correct (and exact) but it seems inconsistent that they should be displayed in E format since they can be represented exactly as 16 digit integers.

The same behavior is observed for negative integers in the complementary range below zero.

NOTE: This is not a hardware representation issue, at least under the assumption of IEEE-754 compliant hardware, which was the case for the examples shown above: An IEEE-754 double precision float exactly represents all integers on [-2^53, +2^53], all of which fit into 16 digits. Approximation error begins at 2^53+1. This can be demonstrated by bumping output_precision to 17:

and so on, all of which is as expected for IEEE-754 hardware.

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by hardy (Posted a comment)
  • -unavailable- added by grepfor (Posted a comment)
  • -unavailable- added by mtmiller (Posted a comment)
  • -unavailable- added by None (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only project members can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 2 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Mon 27 Jan 2014 03:15:57 AM UTCmtmillerStatusNone=>Confirmed
      Release3.6.4=>dev

    Back to the top


    Powered by Savane 3.1-cleanup1