bugGNU Octave - Bugs: bug #30817, Ouput display: truncation vs....

 
 

bug #30817: Ouput display: truncation vs. rounding

Submitted by:  Lukas Reichlin <paramaniac>
Submitted on:  Thu 19 Aug 2010 01:12:34 PM UTC  
 
Category: NoneSeverity: 2 - Minor
Priority: 5 - NormalItem Group: Incorrect Result
Status: Wont FixAssigned to: None
Originator Name: Open/Closed: Closed
Release: 3.2.4Operating System: Mac OS

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

Sat 21 Aug 2010 02:54:06 AM UTC, comment #2:

Octave inherits its output behavior from the C++ standard libraries. In this case, C++ does not round when the last digit is exactly 5. I'm attaching a short program, tst_prec.cpp which demonstrates this behavior.

This is not something that we can fix in Octave.

(file #21267)

Rik <rik5>
Project Administrator
Thu 19 Aug 2010 08:17:34 PM UTC, comment #1:

What you should realize is that when you type 0.999995, the value is already rounded at that point to nearest (or, in general, a close-by) floating-point number. It has to be; there is no other way. So you may be deceived at the spot:

octave:51> x = 9007154218744718 * 2^-53
x = 0.99999
octave:52> x == 0.999995
ans = 1

now with aid of PARI:

? x = 9007154218744718 * 2^-53
%1 = 4503577109372359/4503599627370496
? x - 99999/10^5 < 1 - x
%2 = 1

so, as you see, Octave's result is mathematically correct. Such are the subtleties of binary floating point values.

Jaroslav Hajek <highegg>
Thu 19 Aug 2010 01:12:34 PM UTC, original submission:

I consider the display of result 53 as an inconsistency or a (cosmetic) bug because round (0.5) is 1. Rounding up starts at 0.50, not at 0.51.
(Only the first digit after truncation is significant for rounding)

octave:57> 0.999996
ans = 1.00000
octave:53> 0.999995
ans = 0.99999 # this should be 1.00000 like the previous result
octave:54> 0.9999951
ans = 1.00000
octave:55> 0.9999950
ans = 0.99999 # this should be rounded up too
octave:56> 0.9999949
ans = 0.99999

Results from the "dark side":
0.99995

ans =

1.0000 % this is correct

0.999949

ans =

0.9999

0.999950

ans =

1.0000

0.999951

ans =

1.0000

0.99996

ans =

1.0000

Cheers,
Lukas

Lukas Reichlin <paramaniac>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

Attached Files
file #21267:  tst_prec.cpp added by rik5 (429B - text/x-c++src)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by rik5 (Updated the item)
  • -unavailable- added by highegg (Posted a comment)
  • -unavailable- added by paramaniac (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only project members can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 5 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Fri 27 Aug 2010 12:08:45 PM UTChigheggOpen/ClosedOpen=>Closed
    Sat 21 Aug 2010 02:54:06 AM UTCrik5Attached File-=>Added tst_prec.cpp, #21267
      Severity3 - Normal=>2 - Minor
      StatusNone=>Wont Fix
      Summarytruncation vs. rounding=>Ouput display: truncation vs. rounding

    Back to the top


    Powered by Savane 3.1-cleanup1