bugGNU Octave - Bugs: bug #54029, "Name" is read as Na by...

 
 

bug #54029: "Name" is read as Na by dlmread (and by xlsread) in csv file

Submitter:  Avinoam Kalma <avinoam>
Submitted:  Thu 31 May 2018 10:25:08 PM UTC
   
 
Category:  Octave Function Severity:  2 - Minor
Priority:  3 - Low Item Group:  Inaccurate Result
Status:  Fixed Assigned to:  None
Originator Name:  Avinoam Open/Closed:  * Closed
Release:  * dev Operating System:  * Any
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Thu 26 Jul 2018 06:37:26 PM UTC, comment #5: 

Investigating this bug has led to finding at least three problems with dlmread and scanf.  I pushed a fix for this particular bug here (https://hg.savannah.gnu.org/hgweb/octave/rev/4596a80cbc18).

I don't know if xlsread has been fixed or not since that function resides in a package at the moment.  Marking as fixed and closing report.

Rik <rik5>
Group administrator
Thu 14 Jun 2018 01:21:51 PM UTC, comment #4: 

Thanks for testing.  Matlab is very, very particular about only reading numeric files with dlmread.  That seems a bit too restrictive.  In Octave, skipping headder rows of text is straightforward and this is a common occurrence as many data files have a few lines of introductory text.

So I conclude that it would be nice if Octave were to recognize that "Name" is not Na and substitute the specified EMPTYVAL, but it is also a low priority.  Matlab doesn't even allow for reading a file like nan.csv, and Octave can be made to read it by the trivial workaround of skipping the first text row.


octave:2> dlmread ('nan.csv', ',', 1, 0)
ans =

   1   2   3   4
   5   6   7   8



Rik <rik5>
Group administrator
Thu 14 Jun 2018 11:41:03 AM UTC, comment #3: 

Matlab (2016b):


>> x = dlmread ('nan.csv')
Error using dlmread (line 147)
Mismatch between file and format character vector.
Trouble reading 'Numeric' field from file (row number 1, field number 1) ==> a,b,NaN,c\n

>> x = dlmread ('nane.csv')
Error using dlmread (line 147)
Mismatch between file and format character vector.
Trouble reading 'Numeric' field from file (row number 1, field number 1) ==> a,b,NaNe,c\n

>> x = csvread ('nan.csv')
Error using dlmread (line 147)
Mismatch between file and format character vector.
Trouble reading 'Numeric' field from file (row number 1, field number 1) ==> a,b,NaN,c\n

Error in csvread (line 48)
    m=dlmread(filename, ',', r, c);

>> x = xlsread ('nan.csv')

x =

     1     2     3     4
     5     6     7     8

>> x = xlsread ('nane.csv')

x =

     1     2     3     4
     5     6     7     8


Octave 4.4.0:


>> x = dlmread ('nan.csv')
x =

     0     0   NaN     0
     1     2     3     4
     5     6     7     8

>> x = dlmread ('nane.csv')
x =

     0     0   NaN     0
     1     2     3     4
     5     6     7     8

>> x = csvread ('nan.csv')
x =

     0     0   NaN     0
     1     2     3     4
     5     6     7     8

>> x = xlsread ('nan.csv')
x =

     0     0   NaN     0
     1     2     3     4
     5     6     7     8

>> x = xlsread ('nane.csv')
x =

     0     0   NaN     0
     1     2     3     4
     5     6     7     8


Anonymous
Tue 12 Jun 2018 05:49:41 PM UTC, comment #2: 

Marking as confirmed, but lowering the priority since there are easy workarounds.  dlmread is only to be used for numeric data.  The first line of 'aa.csv' is text and should be skipped entirely by specifying a range to dlmread which excludes the first line.

How does Matlab handle this example file ('nan.csv')?


a,b,NaN,c
1,2,3,4
5,6,7,8


followed by


x = dlmread ('nan.csv')


How does Matlab handle this second example file ('nane.csv')?


a,b,NaNe,c
1,2,3,4
5,6,7,8


followed by


x = dlmread ('nane.csv')



Rik <rik5>
Group administrator
Fri 01 Jun 2018 12:44:57 PM UTC, comment #1: 

Hey, didn't know that xlsread can read .csv files....

(If you do that with the COM interface I suppose it's an upstream bug in Excel, similar to UNO interface with LibreOffice).

Philip Nienhuis <philipnienhuis>
Group Member
Thu 31 May 2018 10:25:08 PM UTC, original submission:  


Example: the file 'aa.csv'


a,b,Name,c
1,2,3,4
5,6,7,8


and:


>> a = dlmread ('aa.csv')
a =

     0     0    NA     0
     1     2     3     4
     5     6     7     8


Avinoam Kalma <avinoam>
Group Member

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by rik5 (Posted a comment)
  • -email is unavailable- added by avinoam (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 7 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2018-07-26 rik5 StatusConfirmed Fixed
        Open/ClosedOpen Closed
    2018-06-12 rik5 Severity3 - Normal 2 - Minor
        Priority5 - Normal 3 - Low
        Item GroupUnexpected Error or Warning Inaccurate Result
        StatusNone Confirmed
        Operating SystemMicrosoft Windows Any

    Back to the top

    Powered by Savane 3.13-4448.
    Corresponding source code