bugGNU Octave - Bugs: bug #50743, textscan produces incorrect...

 
 

bug #50743: textscan produces incorrect results for large (64bit) integers

Submitter:  Ben Lapid <blap>
Submitted:  Thu 06 Apr 2017 10:02:08 PM UTC
   
 
Category:  Octave Function Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Inaccurate Result
Status:  Confirmed Assigned to:  None
Originator Name:  Open/Closed:  * Open
Release:  * dev Operating System:  * GNU/Linux
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Sun 29 Mar 2020 08:00:57 PM UTC, comment #7: 

This is still a problem (3/29/2020).  I added an xtest for this bug so that it will not be forgotten (https://hg.savannah.gnu.org/hgweb/octave/rev/493d46a02edc).

Rik <rik5>
Group administrator
Mon 14 Jan 2019 11:57:56 PM UTC, comment #6: 

My bad. You're right. It's still producing the wrong value in 4.4.1 for me.

Andrew Janke <apjanke>
Mon 14 Jan 2019 10:04:06 PM UTC, comment #5: 

It doesn't work for the original example.  Try


octave:9> z = textscan ('5973459727478852968', '%u64')
z =
{
  [1,1] = 5973459727478853632
}


Rik <rik5>
Group administrator
Sat 12 Jan 2019 03:47:05 AM UTC, comment #4: 

textscan for large ints appears to be working right in 4.4.0 and the latest development version.


octave:7> intmax('int64')
ans = 9223372036854775807
octave:8> textscan('9223372036854775807', '%d64')
ans =
{
  [1,1] = 9223372036854775807
}

octave:9> intmax('uint64')
ans = 18446744073709551615
octave:10> textscan('18446744073709551615', '%u64')
ans =
{
  [1,1] = 18446744073709551615
}

octave:11> ver
----------------------------------------------------------------------
GNU Octave Version: 6.0.0 (hg id: 8b0c9dc0799a)
GNU Octave License: GNU General Public License
Operating System: Darwin 17.7.0 Darwin Kernel Version 17.7.0: Fri Nov  2 20:43:16 PDT 2018; root:xnu-4570.71.17~1/RELEASE_X86_64 x86_64


Is this bug still outstanding?

Andrew Janke <apjanke>
Fri 07 Apr 2017 05:09:13 PM UTC, comment #3: 

I was just looking into that actually. Yes, textscan appears to return values of the right class, but they are actually parsed as doubles.

See the comment here: https://hg.savannah.gnu.org/hgweb/octave/file/56c59b3f9172/libinterp/corefcn/oct-stream.cc#l3317

I still think it's valid to say that dlmread just doesn't do this feature (I agree, a Matlab test to confirm would be nice), and textscan should, but our textscan is currently broken in this regard also.

Mike Miller <mtmiller>
Group Member
Fri 07 Apr 2017 05:03:07 PM UTC, comment #2: 

Can the reporter, or anyone, try the the sample code in Matlab?  My understanding is that dlmread always read into a double matrix and for compatibility Octave needs to do the same.

There could still be an issue, however, with extremely large integer constants.  This should work, but appears to get a differently wrong answer.


fid = fopen ('/tmp/1.csv')
ctave:5> c = textscan (fid, '%u64')
c =
{
  [1,1] = 5973459727478853632
}



Rik <rik5>
Group administrator
Thu 06 Apr 2017 10:21:19 PM UTC, comment #1: 

I don't know for certain, but I believe that's what textscan is for, dlmread should not be that flexible.

If you want simple parsing, where "simple" means the file is delimited columns and all values can be treated as doubles, then use csvread or dlmread. If you want the full flexibility to define how each column should be interpreted, use textscan.

Mike Miller <mtmiller>
Group Member
Thu 06 Apr 2017 10:02:08 PM UTC, original submission:  

Steps to reproduce:
1. (shell) echo 5973459727478852968 > /tmp/1.csv
2. (octave) x = dlmread('/tmp/1.csv', ',', 0, 0)
3. (octave) uint64(x)

>> ans = 5973459727478852608

This issue comes from the fact that dlmread reads doubles (libinterp/corefcn/dlmread.cc) from the text file.
While I understand this behavior in 99% of the cases, and I agree it should be defaulted this way, I do think there should be a way to direct dlmread to a different parse method. See for example scipy's 'genfromtxt' 'dtype' argument.

Ben Lapid <blap>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by apjanke (Posted a comment)
  • -email is unavailable- added by rik5 (Posted a comment)
  • -email is unavailable- added by blap (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 5 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2019-01-15 mtmiller Carbon-CopyRemoved 80942 -
    2017-06-09 rik5 StatusNeed Info Confirmed
        Release4.0.0 dev
        Summarydlmread producing incorrect results for large (64bit) integers textscan produces incorrect results for large (64bit) integers
    2017-04-07 rik5 StatusNone Need Info

    Back to the top

    Powered by Savane 3.13-4448.
    Corresponding source code