bugGNU Octave - Bugs: bug #52867, textscan produces an incorrect...

 
 

bug #52867: textscan produces an incorrect cell array

Submitted by:  Pedro Pena <pedrolpena>
Submitted on:  Fri 12 Jan 2018 02:04:55 AM UTC  
 
Category: Octave FunctionSeverity: 3 - Normal
Priority: 5 - NormalItem Group: Incorrect Result
Status: ConfirmedAssigned to: None
Originator Name: Pedro PenaOpen/Closed: Open
Release: devOperating System: Any

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

(Jump to the original submission Jump to the original submission)

Mon 15 Jan 2018 02:27:13 PM UTC, comment #7:

@Pedro:
A workaround is the following code that works for me to read your file:

Philip Nienhuis <philipnienhuis>
Project Member
Mon 15 Jan 2018 02:20:15 AM UTC, comment #6:

Thank you for pointing me in the right direction.
I've replaced the textscan portion of my code with
a textread conversion.I was not able to produce consistent results with what I was doing and it was really confusing me. After some massaging, I managed to get it to work realizing that there might be a bug in textread or one of its backend functions. I've searched for the bug but I can't seem to find one like it.
I'm going to submit a bug report for textread.

Pedro Pena <pedrolpena>
Sun 14 Jan 2018 11:43:23 PM UTC, comment #5:

O.k., good tip.
I'll implement it that way in the meantime.
Thanks

Pedro Pena <pedrolpena>
Sat 13 Jan 2018 05:00:20 PM UTC, comment #4:

BTW you can try to read the file with textread.m
That still invokes strread.m (that IIRC was textscan's backend in Octave-4.0.0) and takes the same paraneters and format strings.

Philip Nienhuis <philipnienhuis>
Project Member
Fri 12 Jan 2018 07:32:10 PM UTC, comment #3:

The cell array structure is good, so I agree it's
likely being read incorrectly.

In the submitted example a text file is prcoessed by textscan.
This text files contains text representing variable names and the values assigned to the variable names.
textscan should produce a cell matrix containing variable names along with the values for those variables.

cruiseVars{1} should contain the all of the variable names(left of "=").
cruiseVars{2} should contain the values assigned to the variables(right of "=").

This makes it easy to search for values using the variable name.

For example, say I want to retrieve the value assigned to variable "cruise_id" after cruiseVars is created.

I have a function that does this.
https://github.com/pedrolpena/visbeck_ladcp_processing/blob/master/m/misc/get_cruise_variable_value.m

In the above example "AB1705" is indeed the value assigned to variable "cruise_id" this was taken from the following line in the text file.

However when I attempt to extract the value for "cruise_id_prefix", the value returned is "cruise_id_suffix", which
means that the data that should be contained in cruiseVars{1} has been placed in cruiseVars{2}. It's as if textscan "sees" the next line
as being after the "=" on the same line.

This following is what it looks like to textscan.

It looks like the CRLF is being seen as another delimiter (I imagine it does use it as a new line delimiter).
Maybe there is some logic that applies the newline delimiter to the delimiter used in the text tokenization process?
A newline detection next to a delimiter detection issue?

With this in mind I tried one of your test's and I replaced the blank with a character and this made it work as expected even though it failed for you.

the values on the left of the "=" are placed in cruiseVars{1} and the values on the right of "=" are placed in cruiseVars{2} as it should.

Pedro Pena <pedrolpena>
Fri 12 Jan 2018 08:29:13 AM UTC, comment #2:

Furthermore, it makes no difference whether textscan reads from file or from text:

(note I changed file name extension but that shouldn't matter)

and

  • commentstyle (I tried with removing the comment lines),
  • whitespace setting,
  • endofline (your file contains CRLF, I also tried with just LF) and
  • delimiter (changed it into ";")
  • empty fields (I added "a" after the "=" in the "cruise_***fix=" lines)

make no difference either.

I cannot confirm that the cell array is "garbled" (the cell array structure itself looks good), it's just that the file has been read incorrectly, a.o. it consistently skips the line "cruise_id_suffix=".

-> Title adapted

Philip Nienhuis <philipnienhuis>
Project Member
Fri 12 Jan 2018 08:10:06 AM UTC, comment #1:

Confirmed, it looks like it is still the same issue in 4.3.0+ (4.3.0+ tip from Jan. 10, Windows 7).

FTR, Matlab r2018a prerelease confirms that Octave-4.0.2 did it right.

Release -> dev
OS -> Any

Philip Nienhuis <philipnienhuis>
Project Member
Fri 12 Jan 2018 02:04:55 AM UTC, original submission:

textscan under octave 4.2.1 produces a garbled cell array from a text file input but under Octave 4.02 the output is as one
would it expect it to be.

A commented text file(configuration file) with variables and assigned values is read. The delimiter is "=" with "#" read as a comment.

the file 'cruise_params.cfg.tmp'
is first opened and then read with textscan.

The contents are then placed in a cell array.

All works fine under Octave 4.02 but under octave 4.2.1 the cell array is garbled.

I've included the input file "cruise_params.cfg.tmp" for testing and I've included a text file(octaveComparison.txt) with the results from the textscan for each octave version.

Below is a snippet of the output.

Octave 4.02 (correct)
============

Octave 4.21 (incorrect)
===========

I'm not sure if it's a problem with textscan or one of its underlying functions.

This is part of a processing script that runs fine under octave 4.02
https://github.com/pedrolpena/visbeck_ladcp_processing

Pedro Pena <pedrolpena>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

Attached Files
file #42889:  cruise_params.cfg.tmp added by pedrolpena (44KiB - application/octet-stream - files)
file #42890:  octaveComparison.txt added by pedrolpena (11KiB - text/plain - files)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by pedrolpena (Submitted the item)
  • -unavailable- added by pedrolpena (octave 4.2.1 textscan bug)
  • -unavailable- added by pedrolpena (octave 4.2.1 textscan bug)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only project members can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 8 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Fri 12 Jan 2018 08:29:13 AM UTCphilipnienhuisSummarytextscan reads text file using file handle and produces a garbled cell array=>textscan produces an incorrect cell array
    Fri 12 Jan 2018 08:10:06 AM UTCphilipnienhuisStatusNone=>Confirmed
      Release4.2.1=>dev
      Operating SystemGNU/Linux=>Any
    Fri 12 Jan 2018 02:04:55 AM UTCpedrolpenaAttached File-=>Added cruise_params.cfg.tmp, #42889
      Attached File-=>Added octaveComparison.txt, #42890
      Carbon-Copy-=>Added -unavailable-
      Carbon-Copy-=>Added -unavailable-

    Back to the top


    Powered by Savane 3.1-cleanup1