bugGNU Octave - Bugs: bug #31402, dec2base gives incorrect results...

 
 

bug #31402: dec2base gives incorrect results for large integers.

Submitted by:  Gunnar Farnebäck <gf>
Submitted on:  Thu 21 Oct 2010 08:52:18 PM UTC  
 
Category: NoneSeverity: 3 - Normal
Priority: 5 - NormalItem Group: Incorrect Result
Status: FixedAssigned to: None
Originator Name: Gunnar FarnebäckOpen/Closed: Closed
Release: devOperating System: GNU/Linux

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

Sat 30 Oct 2010 07:15:19 PM UTC, comment #1:

I have applied your patch, although I modified it a bit as it didn't work with row vector inputs. Not only does it extend the range of dec2base, but benchmarking shows that it is roughly twice as fast as the previous script. Awesome!

The changeset is in the Mercurial archive: http://hg.savannah.gnu.org/hgweb/octave/rev/51beeeff9027

Rik <rik5>
Project Administrator
Thu 21 Oct 2010 08:52:18 PM UTC, original submission:

Large integers which lose precision in a double may be converted incorrectly by dec2base. E.g.

octave:72> dec2base(uint64(2)^63-1,16)
ans = 8GGFFFFFFFFFFFFF

The attached patch modifies the conversion to use a more robust algorithm, giving the correct result

octave:73> dec2base(uint64(2)^63-1,16)
ans = 7FFFFFFFFFFFFFFF

Gunnar Farnebäck <gf>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

Attached Files

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by rik5 (Posted a comment)
  • -unavailable- added by gf (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only project members can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 3 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Sat 30 Oct 2010 07:15:19 PM UTCrik5StatusNone=>Fixed
      Open/ClosedOpen=>Closed
    Thu 21 Oct 2010 08:52:18 PM UTCgfAttached File-=>Added large_integers_in_dec2base.diff, #21752

    Back to the top


    Powered by Savane 3.1-cleanup1