bugGNU Octave - Bugs: bug #60711, textscan with '.' char read as...

 
 

bug #60711: textscan with '.' char read as float

Submitted by:  Paul Netsaver <netsaver>
Submitted on:  Mon 31 May 2021 01:35:59 PM UTC  
 
Category:  Octave Function Severity:  2 - Minor
Priority:  3 - Low Item Group:  Matlab Compatibility
Status:  Confirmed Assigned to:  None
Originator Name:  Netsaver Paul Open/Closed:  Open
Release:  6.2.0 Operating System:  Any

Add a New Comment (Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

Fri 04 Jun 2021 03:56:25 PM UTC, comment #3: 

Marking as Confirmed since there is a demonstrated Matlab incompatibility.  However, I'm not sure when any developer will have time to take this up.

Rik <rik5>
Project Administrator
Wed 02 Jun 2021 08:55:10 PM UTC, comment #2: 

>>I'm not sure why Matlab decides there are two values here when the delimiter was clearly specified to be the comma ','. 

I think the problem is not the delimiter, it is the dot.
Replace the dot with another char, e.h. 'a':

>> q=textscan([',,a,,3'],'%f','delimiter',',')
q =
{
  [1,1] =
     NaN
     NaN
}

You see that textscan stops when doesn't find a match (this results is the same in Octave and Matlab), while NaN does not stop the scan as it's a match with an EmptyValue.
I think that the true difference is that according to Matlab the dot '.' does not match a float '%f' (therefore stops the scan and does not read the following NaNs), while Octave does match with a NaN float, irrespective of the delimiter.
Probabily Octave starts a matching with '.' waiting for a number (e.g. '.0') and initializes a NaN, then when it finds the limiter and cannot convert to a number leaves the NaN.
Matlab instead understands that '.' alone cannot resolve to a number (even empty) and stops the scan.
By the way I can resolve with a dedicated check, it was only to understand if this was the expected behavior or not.

Thanks and regards,
Netsaver

Paul Netsaver <netsaver>
Wed 02 Jun 2021 12:02:50 AM UTC, comment #1: 

There is a difference with Matlab, but Matlab's approach doesn't make as much sense to me as Octave's.

Take a slightly different example for illustrative purposes.

q=textscan([',,2,,'],'%f','delimiter',',', 'emptyvalue', -1)

I specifically set the EmptyValue option to something besides NaN so the processing of textscan is more obvious.  In both Octave and Matlab this returns

q =
{
  [1,1] =

    -1
    -1
     2
    -1

}

This makes sense to me.  There is a missing value at the start of the string, the second position is also missing, the third value is the number 2, and then then there is another missing value.

Now try reading just a single '.' character.

textscan ('.', '%f')

Both Octave and Matlab return [] for this.

Based on this data, I would expect that ',,.,,' would be treated as EmptyValue EmptyValue EmptyValue EmptyValue which is exactly what Octave does.  I'm not sure why Matlab decides there are two values here when the delimiter was clearly specified to be the comma ','.

Of course, if you have built your textscan parser to rely on this Matlab-specific behavior then it will be a problem.  A simple hack would be to use regular expressions and search&replace bare instances of '.' with something else including possibly nothing.

Rik <rik5>
Project Administrator
Mon 31 May 2021 01:35:59 PM UTC, original submission:  

According to matlab, textscan function "stops when it cannot match formatSpec to the data."
While writing a wrapper for textscan in my application, I could see that the char '.' (alone), when matched with a float with '%f' specifier, is read as 'NaN', letting the scan continue.
In matlab it doesn't match with '%f', stopping the scan at that position.
Here the text in the first parameter position emulates a file with 'eol' chars, but the results were the same with a real file...

q=textscan([',,.',newline,',,'],'%f','delimiter',',')
q{:}
matlab: NaN NaN
octave: q =
{
  [1,1] =

     NaN
     NaN
     NaN
     NaN
     NaN

}

the 3rd nan matches the '.' char
Note that (as with matlab), empty values before the delimiter are converted to NaN, while empty values after the delimiter and before the eol are not converted to nan (but does not stop the scan).

Marked as Matlab compatibility issue, but don't know if wanted or not.

regards,
Netsaver

Paul Netsaver <netsaver>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by rik5 (Posted a comment)
  • -email is unavailable- added by netsaver (Submitted the item)
  • -email is unavailable- added by netsaver
  •  

    Do you think this task is very important?
    If so, you can add your encouragement to it.
    This task has 0 encouragements so far.

    Only project members can vote.

     

     

     

    Follow 5 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2021-06-04 rik5 Severity3 - Normal => 2 - Minor
        Priority5 - Normal => 3 - Low
        StatusNone => Confirmed
    2021-06-02 rik5 Operating SystemMicrosoft Windows => Any
    2021-05-31 netsaver Carbon-Copy- => Added netsaverpaul@gmail.com

    Back to the top


    Powered by Savane 3.6