bugGNU Octave - Bugs: bug #45808, fscanf odd behavior with newlines

 
 

bug #45808: fscanf odd behavior with newlines

Submitted by:  None
Submitted on:  Fri 21 Aug 2015 11:48:51 PM UTC  
 
Category: Octave FunctionSeverity: 3 - Normal
Priority: 5 - NormalItem Group: Matlab Compatibility
Status: FixedAssigned to: John W. Eaton <jwe>
Originator Name: Jordi Burguet-CastellOriginator Email: -unavailable-
Open/Closed: ClosedRelease: 4.0.0
Operating System: Any

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

(Jump to the original submission Jump to the original submission)

Fri 28 Aug 2015 03:53:16 PM UTC, comment #13:

Jordi: Open another bug report about the way strings are displayed (assuming there is not already a report about that). Maybe we should display them as "foo" or 'foo' (depending on whether they are double- or single-quoted strings.

John W. Eaton <jwe>
Project AdministratorIn charge of this item.
Thu 27 Aug 2015 11:27:41 PM UTC, comment #12:

@Rick: It is working now, thanks! I hadn't run "hg update", just "hg pull" (I'm more used to git), so it wasn't applying the last changes. Sorry for the noise, and thank you and John for the fix!

Jordi Burguet Castell <jbc>
Thu 27 Aug 2015 09:54:56 PM UTC, comment #11:

@Jordi: Did you run 'hg update' to get to the latest changeset? Also try running 'run-octave -f' so that no initialization files get read. It's possible there is something in your specific configuration. Otherwise, yes, the example m-file and text file ran fine for me with the patched Octave.

Rik <rik5>
Project Administrator
Thu 27 Aug 2015 09:17:13 PM UTC, comment #10:

Hi,

Thanks for looking into it and the fix. Does it really pass the original example for you? I just tried it and still find the same result:
ans = 1
ans = y

(By the way, I find it confusing that strings are not quoted now, but that's a different story.)

I've got the repository updated until this log:

changeset: 20465:44eb1102f8a8
bookmark: @
tag: tip
user: John W. Eaton <jwe@octave.org>
date: Wed Aug 26 16:05:49 2015 -0400
summary: don't recycle scanf format string if all conversions are done (bug #45808)

and I compiled it in debian sid, without any octave package installed and with a recent version of the repository (until the commit mentioned above), and just to be sure I "make clean" and "make distclean"-ed before compiling again.

Jordi Burguet Castell <jbc>
Thu 27 Aug 2015 04:32:59 PM UTC, comment #9:

Passes all of the test suite now as well as the example in this bug report. Marking as Fixed and closing report.

Rik <rik5>
Project Administrator
Thu 27 Aug 2015 03:30:44 PM UTC, comment #8:

I pushed the following modified version of the change I proposed earlier. The io tests all succeed for me now.

http://hg.savannah.gnu.org/hgweb/octave/rev/44eb1102f8a8

John W. Eaton <jwe>
Project AdministratorIn charge of this item.
Wed 26 Aug 2015 10:07:13 PM UTC, comment #7:

Running 'make check' with the patch applied produces 12 failures, so the patch seems to have some side effects. The output from fntests.log is shown below.

Rik <rik5>
Project Administrator
Wed 26 Aug 2015 09:02:25 PM UTC, comment #6:

The patch works for me. I think you can go ahead and push it to Mercurial.

Rik <rik5>
Project Administrator
Wed 26 Aug 2015 08:08:22 PM UTC, comment #5:

I think the attached patch fixes the problem. There was already some logic to handle not recycling through the list but it didn't correctly handle the case of literal or whitespace conversions at the beginning and end of a format with a limited number of conversions.

(file #34748)

John W. Eaton <jwe>
Project AdministratorIn charge of this item.
Wed 26 Aug 2015 05:42:35 PM UTC, comment #4:

I went back to first principles and assume that fscanf should behave as the low-level C function.

I've attached a simple example, fscanf.c, and an accompanying data file, fscanf.txt. Use 'gcc fscanf.c' to compile and then run a.out.

The newline character is counted as whitespace and eats as much as possible. In the example file I have a line following the first that is just made of of spaces, a tab, and a completing newline. All of it gets eaten by fscanf. So the Matlab result is correct.

At least at the C level, I checked and it seems that a partially failing template still eats whitespace following the last successfuly read parameter.

(file #34740, file #34741)

Rik <rik5>
Project Administrator
Wed 26 Aug 2015 04:37:58 PM UTC, comment #3:

I don't think the problem is really with newlines. It's that a limit on the number of requested values is specified but Octave is cycling through the format string again. So it skips the whitespace (newline) and then the ";" on the second line of the file matches the beginning of the format string but there is a failure to read the "y" with "%d". Only when that format element fails to match does it return and move on to the next call to fscanf.

So I think the solution is to not recycle a format string if a fixed limit on the number of requested values is supplied. But I'm not sure of exactly what is supposed to happen. For example, does

stop reading immediately after the first "%d" conversion, or does it also grab any literal text and whitespace following that conversion? What about something like

Does the limit apply immediately to values read or does it also process the second (skipped) number?

I don't quite understand the rules yet.

John W. Eaton <jwe>
Project AdministratorIn charge of this item.
Wed 26 Aug 2015 06:46:58 AM UTC, comment #2:

Thanks for the input. That would work for this particular example, but I'm finding the problem as part of a larger script that parses a more-or-less complex text file. I've tried several changes, including your suggestion, and it breaks the parsing later on, one way or another, in a non-trivial way.

The fact is that matlab can easily parse those files, and I've given up for the moment trying to have octave parse them too with the same code, because it got so complicated.

In any case, octave is behaving different than matlab there, so it can be fixed. Unfortunately I don't know about a workaround that worked for me, but I can wait until it's fixed in octave.

Jordi Burguet Castell <jbc>
Tue 25 Aug 2015 04:04:29 AM UTC, comment #1:

Confirmed. There does seem to be an issue with newlines in the format string and this was on a Linux platform.

Could you try the attached workaround? It simply removes the newline from the format string. I believe both Matlab and Octave will reset their internal positioning counter at the end of each line and this should work on both platforms.

(file #34718)

Rik <rik5>
Project Administrator
Fri 21 Aug 2015 11:48:51 PM UTC, original submission:

When there is a newline ('\n') in the format string for fscanf, it seems that further calls to fscanf don't read from where it is supposed to.

For example, running the attached program produces:

In matlab:
ans = 1
ans = ';'

In octave:
ans = 1
ans = 'y'

I have tried it in windows, but may be present in other systems too.

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

Attached Files
file #34748:  diffs.txt added by jwe (2KiB - text/plain)
file #34741:  fscanf.txt added by rik5 (18B - text/plain)
file #34740:  fscanf.c added by rik5 (568B - text/x-csrc)
file #34718:  fscanf_workaround.m added by rik5 (85B - text/x-objcsrc)
file #34692:  matlab_octave_diff.m added by None (87B - text/plain - the .m file reads data from the .txt file)
file #34693:  matlab_octave_diff.txt added by None (10B - text/plain - the .m file reads data from the .txt file)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by jwe (Posted a comment)
  • -unavailable- added by jbc (Posted a comment)
  • -unavailable- added by rik5 (Updated the item)
  • -unavailable- added by None (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only project members can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 13 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Thu 27 Aug 2015 04:32:59 PM UTCrik5StatusReady For Test=>Fixed
      Open/ClosedOpen=>Closed
    Thu 27 Aug 2015 03:30:44 PM UTCjweStatusPatch Submitted=>Ready For Test
    Wed 26 Aug 2015 09:02:25 PM UTCrik5Assigned toNone=>jwe
      Operating SystemMicrosoft Windows=>Any
    Wed 26 Aug 2015 08:08:34 PM UTCjweStatusConfirmed=>Patch Submitted
    Wed 26 Aug 2015 08:08:22 PM UTCjweAttached File-=>Added diffs.txt, #34748
    Wed 26 Aug 2015 05:42:35 PM UTCrik5Attached File-=>Added fscanf.c, #34740
      Attached File-=>Added fscanf.txt, #34741
    Tue 25 Aug 2015 04:04:29 AM UTCrik5Attached File-=>Added fscanf_workaround.m, #34718
      StatusNone=>Confirmed
    Fri 21 Aug 2015 11:48:51 PM UTCNoneAttached File-=>Added matlab_octave_diff.m, #34692
      Attached File-=>Added matlab_octave_diff.txt, #34693

    Back to the top


    Powered by Savane 3.1-cleanup1