bugGNU Octave - Bugs: bug #50743, dlmread producing incorrect...

 
 

bug #50743: dlmread producing incorrect results for large (64bit) integers

Submitted by:  Ben Lapid <blap>
Submitted on:  Thu Apr 6 22:02:08 2017  
 
Category: Octave FunctionSeverity: 3 - Normal
Priority: 5 - NormalItem Group: Inaccurate Result
Status: Need InfoAssigned to: None
Originator Name: Open/Closed: Open
Release: 4.0.0Operating System: GNU/Linux

Add a New Comment (Rich MarkupRich Markup):
   

You are not logged in

Please log in, so followups can be emailed to you.

 

Fri Apr 7 17:09:13 2017, comment #3:

I was just looking into that actually. Yes, textscan appears to return values of the right class, but they are actually parsed as doubles.

See the comment here: https://hg.savannah.gnu.org/hgweb/octave/file/56c59b3f9172/libinterp/corefcn/oct-stream.cc#l3317

I still think it's valid to say that dlmread just doesn't do this feature (I agree, a Matlab test to confirm would be nice), and textscan should, but our textscan is currently broken in this regard also.

Mike Miller <mtmiller>
Project Administrator
Fri Apr 7 17:03:07 2017, comment #2:

Can the reporter, or anyone, try the the sample code in Matlab? My understanding is that dlmread always read into a double matrix and for compatibility Octave needs to do the same.

There could still be an issue, however, with extremely large integer constants. This should work, but appears to get a differently wrong answer.

Rik <rik5>
Project Administrator
Thu Apr 6 22:21:19 2017, comment #1:

I don't know for certain, but I believe that's what textscan is for, dlmread should not be that flexible.

If you want simple parsing, where "simple" means the file is delimited columns and all values can be treated as doubles, then use csvread or dlmread. If you want the full flexibility to define how each column should be interpreted, use textscan.

Mike Miller <mtmiller>
Project Administrator
Thu Apr 6 22:02:08 2017, original submission:

Steps to reproduce:
1. (shell) echo 5973459727478852968 > /tmp/1.csv
2. (octave) x = dlmread('/tmp/1.csv', ',', 0, 0)
3. (octave) uint64(x)

>> ans = 5973459727478852608

This issue comes from the fact that dlmread reads doubles (libinterp/corefcn/dlmread.cc) from the text file.
While I understand this behavior in 99% of the cases, and I agree it should be defaulted this way, I do think there should be a way to direct dlmread to a different parse method. See for example scipy's 'genfromtxt' 'dtype' argument.

Ben Lapid <blap>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by rik5 (Posted a comment)
  • -unavailable- added by mtmiller (Posted a comment)
  • -unavailable- added by blap (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only project members can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follows 1 latest change.

    Date Changed By Updated Field Previous Value => Replaced By
    Fri Apr 7 17:03:07 2017rik5StatusNone=>Need Info

    Back to the top


    Powered by Savane 3.1-cleanup1