bugGNU Octave - Bugs: bug #50743, textscan produces incorrect...

 
 

bug #50743: textscan produces incorrect results for large (64bit) integers

Submitted by:  Ben Lapid <blap>
Submitted on:  Thu 06 Apr 2017 10:02:08 PM UTC  
 
Category: Octave FunctionSeverity: 3 - Normal
Priority: 5 - NormalItem Group: Inaccurate Result
Status: ConfirmedAssigned to: None
Originator Name: Open/Closed: Open
Release: devOperating System: GNU/Linux

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

Fri 07 Apr 2017 05:09:13 PM UTC, comment #3:

I was just looking into that actually. Yes, textscan appears to return values of the right class, but they are actually parsed as doubles.

See the comment here: https://hg.savannah.gnu.org/hgweb/octave/file/56c59b3f9172/libinterp/corefcn/oct-stream.cc#l3317

I still think it's valid to say that dlmread just doesn't do this feature (I agree, a Matlab test to confirm would be nice), and textscan should, but our textscan is currently broken in this regard also.

Mike Miller <mtmiller>
Project Administrator
Fri 07 Apr 2017 05:03:07 PM UTC, comment #2:

Can the reporter, or anyone, try the the sample code in Matlab? My understanding is that dlmread always read into a double matrix and for compatibility Octave needs to do the same.

There could still be an issue, however, with extremely large integer constants. This should work, but appears to get a differently wrong answer.

Rik <rik5>
Project Administrator
Thu 06 Apr 2017 10:21:19 PM UTC, comment #1:

I don't know for certain, but I believe that's what textscan is for, dlmread should not be that flexible.

If you want simple parsing, where "simple" means the file is delimited columns and all values can be treated as doubles, then use csvread or dlmread. If you want the full flexibility to define how each column should be interpreted, use textscan.

Mike Miller <mtmiller>
Project Administrator
Thu 06 Apr 2017 10:02:08 PM UTC, original submission:

Steps to reproduce:
1. (shell) echo 5973459727478852968 > /tmp/1.csv
2. (octave) x = dlmread('/tmp/1.csv', ',', 0, 0)
3. (octave) uint64(x)

>> ans = 5973459727478852608

This issue comes from the fact that dlmread reads doubles (libinterp/corefcn/dlmread.cc) from the text file.
While I understand this behavior in 99% of the cases, and I agree it should be defaulted this way, I do think there should be a way to direct dlmread to a different parse method. See for example scipy's 'genfromtxt' 'dtype' argument.

Ben Lapid <blap>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by rik5 (Posted a comment)
  • -unavailable- added by mtmiller (Posted a comment)
  • -unavailable- added by blap (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only project members can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 4 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Fri 09 Jun 2017 08:37:15 PM UTCrik5StatusNeed Info=>Confirmed
      Release4.0.0=>dev
      Summarydlmread producing incorrect results for large (64bit) integers=>textscan produces incorrect results for large (64bit) integers
    Fri 07 Apr 2017 05:03:07 PM UTCrik5StatusNone=>Need Info

    Back to the top


    Powered by Savane 3.1-cleanup1