bugGNU Octave - Bugs: bug #51231, textscan skips line ending

 
 

bug #51231: textscan skips line ending

Submitted by:  Julius Popp <juliuspopp>
Submitted on:  Tue 13 Jun 2017 01:21:38 PM UTC  
 
Category: Octave FunctionSeverity: 3 - Normal
Priority: 5 - NormalItem Group: Matlab Compatibility
Status: ConfirmedAssigned to: None
Originator Name: Open/Closed: Open
Release: 4.2.1Operating System: Microsoft Windows

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

Mon 26 Jun 2017 09:08:50 PM UTC, comment #4:

So "help textscan" says:

_N
Read until the first of two conditions occurs:
1) the format has been processed N times, or
2) N lines of the input have been processed._

I am not sure if the second of those is Matlab compatible. Maybe that is a relict from the time textscan was implemented as an .m file (but I even doubt that, I think it is plain wrong).

In the mean time I've checked with Matlab and indeed that behaves differently than Octave. Admittedly a bit of a surprise for me :-)

The issue may boil down to incorrect default whitespace and -delimiter settings. There's another bug report that I suspect also relates to that.
Nowadays textscan is a binary function and my C++ proficiency is a bit lacking - when textscan was an .m file I could do something about it.

Status -> Confirmed
Item group -> Matlab compatibility

Philip Nienhuis <philipnienhuis>
Project Member
Mon 26 Jun 2017 03:04:01 PM UTC, comment #3:

Thanks for the explanation, I'll be able to adapt to the current version now.

I'd like to ask about the documentation as I understand it that textscan does handle lines.

Octave 4.0.3 documentation:

-- Function File: C = textscan (FID, FORMAT, N)
...
"When reading from file, N specifies the number of data lines to read; in this sense it differs slightly from the format repeat count in strread."

Octave 4.2.1 documentation and https://octave.sourceforge.io/octave/function/textscan.html:

C = textscan (fid, format, repeat)
...
The optional numeric argument REPEAT can be used for limiting the
number of items read:
...
N
Read until the first of two conditions occurs:
1) the format has been processed N times, or
2) N lines of the input have been processed.

p.s. I do not have easy access to a Matlab system right now.

p.p.s.: I tried to build a workaround which unfortunately does skip empty lines (worked around that as well):

line = textscan(fid, '%s', 1, "Delimiter", "\r\n");

words = strsplit(line{1}{1}, {" ", "\t"});

Julius Popp <juliuspopp>
Sat 17 Jun 2017 09:20:57 PM UTC, comment #2:

I am not sure this is a bug. I rather suspect you mix up the meaning of textscan's format string and the number of data per line in a data file.

A format repeat count instructs textscan to invoke the format string the specified number of times, no matter how many lines the format would cover.
(The format string may even cover just part of a "line" in a data file.)
IOW, the format repeat count does NOT specify to read the specified number of lines.

FYI, textscan() interprets a data file as just one long sequence of characters and has no notion of "lines" in a data file; line endings (EOL) are treated as ordinary delimiters by default.

A textscan format string just specifies how many data fields are to be read and turned into a corresponding number of output data columns. There is no implicit connection between number of data fields in the format string and the number of data per line in the data file.
So in your example textscan does seem to do the right thing. It invokes the format string once, and that format string tells textscan to read three data fields and and return them in three separate output data columns.

Only if you'd specify a format specifier like '%*[^\n]' (= skip until the first end-of-line after the last read data field) some notion of data file lines comes in sight. But note that even in that case textscan may still read several data lines per format string.

This is different than e.g., dlmread() that does process files line by line.

As I've maintained most of textscan.m and strread.m in the last years I feel qualified to say that I'd rather expect the old textscan.m (or rather, its work horse strread.m) to do it wrong.

A comparison with Matlab would help here. What does Matlab return?

Philip Nienhuis <philipnienhuis>
Project Member
Tue 13 Jun 2017 02:38:16 PM UTC, comment #1:

note: this is a regression from the rewrite of textscan for Octave 4.2. This bug is not there in Octave 4.0.3.

Julius Popp <juliuspopp>
Tue 13 Jun 2017 01:21:38 PM UTC, original submission:

M = [11 12; 21 22; 31 32];
filename = 'testAscii.txt';
dlmwrite(filename, M, ' ');

fid = fopen(filename);
data = textscan(fid, '%s %s %s', 1) % should only read 1 line

data =
{
[1,1] =
{
[1,1] = 11
}
[1,2] =
{
[1,1] = 12
}
[1,3] =
{
[1,1] = 21
}
}

>>

Julius Popp <juliuspopp>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by mtmiller (Updated the item)
  • -unavailable- added by juliuspopp (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only project members can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 4 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Mon 14 Aug 2017 05:53:06 PM UTCmtmillerCategoryNone=>Octave Function
    Mon 26 Jun 2017 09:08:50 PM UTCphilipnienhuisItem GroupNone=>Matlab Compatibility
      StatusNeed Info=>Confirmed
    Sat 17 Jun 2017 09:20:57 PM UTCphilipnienhuisStatusNone=>Need Info

    Back to the top


    Powered by Savane 3.1-cleanup1