bugGNU Octave - Bugs: bug #46493, scanf with %i and out of range...

 
 

bug #46493: scanf with %i and out of range octal does not set errmsg

Submitted by:  None
Submitted on:  Sat 21 Nov 2015 06:02:43 PM UTC  
 
Category: Octave FunctionSeverity: 2 - Minor
Priority: 3 - LowItem Group: Matlab Compatibility
Status: PostponedAssigned to: None
Originator Name: Richard SuchlandOriginator Email: -unavailable-
Open/Closed: OpenRelease: dev
Operating System: Any

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

(Jump to the original submission Jump to the original submission)

Mon 01 Feb 2016 10:14:59 AM UTC, comment #13:

Rik, I didn't notice that those variables were private until I tried writing a patch...

My understanding is that it is currently not possible to have a warning that sets the state without issuing a message, which is what is really wanted here. I think the best solution is to address bug #41028 ("warning off" should still set state, for Matlab compatibility) by having three states for each warning: "on" (message and set state), "off" (set state, but don't message), "disabled" or similar (neither message nor set state). Then we could have a warning for this that is "off" by default.

Lachlan Andrew <lachlan>
Project Member
Sun 31 Jan 2016 04:39:33 AM UTC, comment #12:

The issue is that Vlast_warning_id and Vlast_warning_message are meant to be private variables only modified through explicit calls to error and warning. If we have functions changing internal state through backdoor mechanisms then the code can become unstable. Worse, the resulting problems can be very hard to debug.

On the other hand, maybe it would be okay to call warning and then check, as you suggest, for both a warning or os.error being set.

Rik <rik5>
Project Administrator
Fri 29 Jan 2016 11:36:02 PM UTC, comment #11:

How about setting Vlast_warning_id and Vlast_warning_message, and checking them as well as os.error() in Ffscanf?

Depending on the resolution of bug #41028, the cleanest thing would just be to have a warning that defaults to "off, but set state", but until that is resolved, it would be possible just to set Vlast_warning_id and Vlast_warning_message directly.

Lachlan Andrew <lachlan>
Project Member
Wed 20 Jan 2016 02:59:28 AM UTC, comment #10:

I found the point in the code that needs to be modified. It is in corefcn/oct-stream.cc. I added the following at line 1155:

Unfortunately, there is no easy way to get at the error function of the octave-stream which in order to set the errmsg variable. So, this probably won't get fixed.

Rik <rik5>
Project Administrator
Mon 07 Dec 2015 06:52:16 PM UTC, comment #9:

Reopening this report after a more thorough check with Matlab.

Matlab r2014a (empty lines removed):

whereas Octave-4.1.0+ gives:

So Matlab, when explicitly queried using the ERRMSG output arg, informs one that it found illegal characters, while Octave just remains silent.

Release => dev
Item group => Matlab Compatibility

Philip Nienhuis <philipnienhuis>
Project Member
Wed 25 Nov 2015 03:15:44 PM UTC, comment #8:

I think that follows the conventions of the standard C scanf function as well.

Regardless, solution for OP found, Octave working as expected, moving on :)

Mike Miller <mtmiller>
Project Administrator
Wed 25 Nov 2015 02:07:07 PM UTC, comment #7:

@Mike:
Whether it's a bug or not might be debatable.
All we can say is that Octave is compatible with Matlab (but perhaps bug-for-bug compatible).
IMO it can be argued that "09" should also be something that "breaks the recognition of the number as octal" (as per comment #3).

But Matlab itself has the final word:
In table Input Arguments of http://nl.mathworks.com/help/matlab/ref/sscanf.html: "... If initial digit is 0, it is base 8."

I'd conclude that Matlab's base detection logic is a bit quick and dirty, but we have to live with it.

Philip Nienhuis <philipnienhuis>
Project Member
Tue 24 Nov 2015 04:06:30 PM UTC, comment #6:

Based on comment #5, it looks like Octave has the correct behavior, so no bug here.

Mike Miller <mtmiller>
Project Administrator
Tue 24 Nov 2015 01:29:35 PM UTC, comment #5:

Just stumble over this bugreport, here matlabs result from Rik's suggested test of comment #2:

sscanf('"09"', '"%i"')
ans =
0
sscanf('"079"', '"%i"')
ans =
7

Anselm Köhler <ansi512>
Mon 23 Nov 2015 01:51:27 AM UTC, comment #4:

I replaced all occurrences of %i %d per the suggested workaround and using it resulted in correct reading of all data in my dataset!
Thanks

Richard Suchland <suchland>
Mon 23 Nov 2015 12:49:05 AM UTC, comment #3:

Confirmed using Doug's simplified example from comment #2.

As a simple workaround, if you know that the numbers are in decimal format then you can use either %u for unsigned or %d for optionally signed integers. The following format works

The trouble seems to be that %i must recognize decimal, octal, and hexadecimal. In this case, the first character of the number is a '0' which seems to trigger the octal conversion format. But there is no valid octal number for 8 and 9 so it returns '0' for that conversion.

On the other hand, anything you do which breaks the recognition of the number as octal will cause it to work. For example,

It would be interesting to know if Matlab gets this right. If anyone has access, could they run the following two tests?

Rik <rik5>
Project Administrator
Sat 21 Nov 2015 11:49:28 PM UTC, comment #2:

more info

From this we can see that 08 and 09 are wrong but 8 and 9 are ok
and 01 02 03 04 05 06 07 are ok also

Doug Stewart <dastew>
Sat 21 Nov 2015 06:59:37 PM UTC, comment #1:

Here is a shorter version that shows the problem.

Doug Stewart <dastew>
Sat 21 Nov 2015 06:02:43 PM UTC, original submission:

While developing a script to read and plot weather station data from a CSV file using fscanf, I got an unexpected result. After some analysis I tracked it to fscanf reading a
particular reacord. It didn't seem to be a error in the data record and my orignial thought was an issue with the size of the data (this occurred on the 4324th record in the
CSV file). Then I tried reading in the data as bytes converting to characters and using sscanf, but got the same result. Then I cleared memory and executed the following
standalone script which replicated the issue without the concern for memory. The problem is that fscanf did not read all of the "bad" record data.

v =

Columns 1 through 14:

11 7 2015 6 24 0 80 77 46 46 48 69.2 99 46.2

Columns 15 through 28:

28.821 13.5 11 8 2015 11 3 0 65 77 6.8351 42 7.4565 270

Column 29:

9.3206
ans =

1 29

v =

Columns 1 through 14:

11 7 2015 6 36 0 80 77 47 48 48 69.6 96 47.8

Columns 15 through 21:

28.821 13.47 11 7 2015 5 0

>> size(v)

ans =

1 21

>>

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by philipnienhuis
  • -unavailable- added by lachlan (Posted a comment)
  • -unavailable- added by mtmiller (Posted a comment)
  • -unavailable- added by ansi512 (Posted a comment)
  • -unavailable- added by suchland (Posted a comment)
  • -unavailable- added by rik5 (Posted a comment)
  • -unavailable- added by dastew (Posted a comment)
  • -unavailable- added by None (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only project members can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 14 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Wed 23 Nov 2016 05:22:31 AM UTCrik5Severity3 - Normal=>2 - Minor
    Thu 21 Apr 2016 09:15:09 PM UTCphilipnienhuisCarbon-Copy-=>Added philipnienhuis
    Wed 20 Apr 2016 09:56:53 PM UTCmtmillerPriority5 - Normal=>3 - Low
      StatusNone=>Postponed
      SummaryIncorrect result from fscanf and scanf with %i format=>scanf with %i and out of range octal does not set errmsg
    Mon 07 Dec 2015 06:52:16 PM UTCphilipnienhuisItem GroupIncorrect Result=>Matlab Compatibility
      StatusInvalid=>None
      Open/ClosedClosed=>Open
      Release4.0.0=>dev
    Tue 24 Nov 2015 04:06:30 PM UTCmtmillerStatusConfirmed=>Invalid
      Open/ClosedOpen=>Closed
    Mon 23 Nov 2015 12:49:05 AM UTCrik5StatusNone=>Confirmed
      Operating SystemMicrosoft Windows=>Any
      SummaryIncorrect result from fscanf and scanf=>Incorrect result from fscanf and scanf with %i format

    Back to the top


    Powered by Savane 3.1-cleanup1