bugGNU Octave - Bugs: bug #57612, textscan omits final column value...

 
 

bug #57612: textscan omits final column value for CSV is value is empty

Submitted by:  Will Carhart <wcarhart>
Submitted on:  Thu 16 Jan 2020 06:50:04 PM UTC  
 
Category:  Octave Function Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Incorrect Result
Status:  Confirmed Assigned to:  None
Originator Name:  Will Carhart Open/Closed:  Open
Release:  dev Operating System:  GNU/Linux

Add a New Comment (Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

Fri 17 Jan 2020 02:34:38 AM UTC, comment #2: 

Just to acknowledge the title of this bug report.  Octave 5.1.0, 5.1.90, and dev and Matlab R2019a behave the same for the example with final CSV value:

newline = @() sprintf('\n');
str = ['102,19700101,,0.485,42', newline(), '111,19700101,,0.48,42']
C   = textscan (str, '%s%f%q%f%q', 'Delimiter', ',')

Kai Torben Ohlhus <siko1056>
Project Member
Fri 17 Jan 2020 02:27:05 AM UTC, comment #1: 

Thank you for your bug report.  I created an easier equivalent code to reproduce in Octave and Matlab:

newline = @() sprintf('\n');
str = ['102,19700101,,0.485,', newline(), '111,19700101,,0.48,']
C   = textscan (str, '%s%f%q%f%q', 'Delimiter', ',')

Your expected output is exactly that of Matlab R2019a where C{1}{1} and C{1}{2} are "char-arrays".

In Octave 5.1.0, 5.1.90, and dev the output looks gibberish as in comment #0.  Thus changing to dev.

Kai Torben Ohlhus <siko1056>
Project Member
Thu 16 Jan 2020 06:50:04 PM UTC, original submission:  

Running Octave 4.2.2 on Ubuntu 16.04.

textscan will skip the last column value in a CSV file if the column value is empty. For example, consider this code snippet:

lineFormat = [repmat('%s',1,1), ...
         repmat('%f',1,1), ...
         repmat('%q',1,1), ...
         repmat('%f',1,1), ...
         repmat('%q',1,1)];
C = textscan(fid, lineFormat, 'Delimiter', ',')

With the following CSV data:

102,19700101,,0.485,
111,19700101,,0.48,

I would expect the result of textscan to be:

C =
{
  [1,1] =
  {
    [1,1] = 102
    [2,1] = 111
  }
  [1,2] =
     19700101
     19700101
  [1,3] =
  {
    [1,1] =
    [2,1] =
  }
  [1,4] =
     0.48500
     0.48000
  [1,5] =
  {
    [1,1] =
    [2,1] =
  }
}

However, in reality Octave's textscan produces the following:

C =
{
  [1,1] =
  {
    [1,1] = 102
    [2,1] = 19700101
  }
  [1,2] =

     1.9700e+07
            NaN

  [1,3] =
  {
    [1,1] =
    [2,1] = 0.48
  }
  [1,4] =

       0.48500
     110.00000

  [1,5] =
  {
    [1,1] = 111
    [2,1] = 19700101
  }
}

Note that all of the data is shifted by one cell because the last column value for each line in the CSV is empty. This is not the behavior in MATLAB. In MATLAB, textscan can parse the above CSV data as expected.

I have tried the options 'MultipleDelimsAsOne'=1 and 'EndOfLine'='\n' and ='\r', but neither changed the behavior.

Is this expected? If so, is there a way to configure Octave's textscan to perform like MATLAB's?

Will Carhart <wcarhart>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by siko1056 (Posted a comment)
  • -email is unavailable- added by wcarhart (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can add your encouragement to it.
    This task has 0 encouragements so far.

    Only project members can vote.

     

     

     

    Follow 2 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2020-01-17 siko1056 StatusNone => Confirmed
        Release4.2.2 => dev

    Back to the top


    Powered by Savane 3.5