bugGNU Octave - Bugs: bug #46493, scanf with %i and out of range...


bug #46493: scanf with %i and out of range octal does not set errmsg

Submitted by:  None
Submitted on:  Sat 21 Nov 2015 06:02:43 PM UTC  
Category:  Octave Function Severity:  2 - Minor
Priority:  3 - Low Item Group:  Matlab Compatibility
Status:  Fixed Assigned to:  None
Originator Name:  Richard Suchland Originator Email:  -email is unavailable-
Open/Closed:  Closed Release:  dev
Operating System:  Any

Add a New Comment (Rich Markup)

You are not logged in

Please log in, so followups can be emailed to you.


( Jump to the original submission)

Sat 08 Jun 2019 03:16:21 AM UTC, comment #14: 

This has been fixed by this changeset

changeset:   27137:30f53f7a7293
user:        Rik <rik@octave.org>
date:        Mon Jun 03 09:04:41 2019 -0700
summary:     Return correct error messages on octave-streams when scanf fails (bug #56396)

Closing report.

Rik <rik5>
Project Administrator
Mon 01 Feb 2016 10:14:59 AM UTC, comment #13: 

Rik, I didn't notice that those variables were private until I tried writing a patch...

My understanding is that it is currently not possible to have a warning that sets the state without issuing a message, which is what is really wanted here.  I think the best solution is to address bug #41028 ("warning off" should still set state, for Matlab compatibility) by having three states for each warning: "on" (message and set state), "off" (set state, but don't message), "disabled" or similar (neither message nor set state).  Then we could have a warning for this that is "off" by default.

Lachlan Andrew <lachlan>
Sun 31 Jan 2016 04:39:33 AM UTC, comment #12: 

The issue is that Vlast_warning_id and Vlast_warning_message are meant to be private variables only modified through explicit calls to error and warning.  If we have functions changing internal state through backdoor mechanisms then the code can become unstable.  Worse, the resulting problems can be very hard to debug.

On the other hand, maybe it would be okay to call warning and then check, as you suggest, for both a warning or os.error being set.

Rik <rik5>
Project Administrator
Fri 29 Jan 2016 11:36:02 PM UTC, comment #11: 

How about setting  Vlast_warning_id and Vlast_warning_message, and checking them as well as os.error() in  Ffscanf?

Depending on the resolution of bug #41028, the cleanest thing would just be to have a warning that defaults to "off, but set state", but until that is resolved, it would be possible just to set Vlast_warning_id and Vlast_warning_message directly.

Lachlan Andrew <lachlan>
Wed 20 Jan 2016 02:59:28 AM UTC, comment #10: 

I found the point in the code that needs to be modified.  It is in corefcn/oct-stream.cc.  I added the following at line 1155:

else if (c2 == '8' || c2 == '9')
  // FIXME: Would like to set error state on octave stream.
  // See bug #46493.  But only std::istream is input to fcn
  // error ("internal failure to match octal format");
  ref = 0;

Unfortunately, there is no easy way to get at the error function of the octave-stream which in order to set the errmsg variable.  So, this probably won't get fixed.

Rik <rik5>
Project Administrator
Mon 07 Dec 2015 06:52:16 PM UTC, comment #9: 

Reopening this report after a more thorough check with Matlab.

Matlab r2014a (empty lines removed):

>> [VAL, COUNT, ERRMSG] = sscanf('''09''', '''%i''')
Matching failure in format.
>> [VAL, COUNT, ERRMSG] = sscanf('"079"', '"%i"')
Matching failure in format.

whereas Octave-4.1.0+ gives:

>> [VAL, COUNT, ERRMSG] = sscanf('''09''', '''%i''')
VAL = 0
COUNT =  1
>> [VAL, COUNT, ERRMSG] = sscanf('"079"', '"%i"')
VAL =  7
COUNT =  1

So Matlab, when explicitly queried using the ERRMSG output arg, informs one that it found illegal characters, while Octave just remains silent.

Release    => dev
Item group => Matlab Compatibility

Philip Nienhuis <philipnienhuis>
Project Member
Wed 25 Nov 2015 03:15:44 PM UTC, comment #8: 

I think that follows the conventions of the standard C scanf function as well.

Regardless, solution for OP found, Octave working as expected, moving on :)

Mike Miller <mtmiller>
Project Administrator
Wed 25 Nov 2015 02:07:07 PM UTC, comment #7: 

Whether it's a bug or not might be debatable.
All we can say is that Octave is compatible with Matlab (but perhaps bug-for-bug compatible).
IMO it can be argued that "09" should also be something that "breaks the recognition of the number as octal" (as per comment #3).

But Matlab itself has the final word:
In table Input Arguments of http://nl.mathworks.com/help/matlab/ref/sscanf.html: "... If initial digit is 0, it is base 8."

I'd conclude that Matlab's base detection logic is a bit quick and dirty, but we have to live with it.

Philip Nienhuis <philipnienhuis>
Project Member
Tue 24 Nov 2015 04:06:30 PM UTC, comment #6: 

Based on comment #5, it looks like Octave has the correct behavior, so no bug here.

Mike Miller <mtmiller>
Project Administrator
Tue 24 Nov 2015 01:29:35 PM UTC, comment #5: 

Just stumble over this bugreport, here matlabs result from Rik's suggested test of comment #2:

sscanf('"09"', '"%i"')
ans =
sscanf('"079"', '"%i"')
ans =

Anselm Köhler <ansi512>
Mon 23 Nov 2015 01:51:27 AM UTC, comment #4: 

I replaced all occurrences of %i %d per the suggested workaround and using it resulted in correct reading of all data in my dataset!

Richard Suchland <suchland>
Mon 23 Nov 2015 12:49:05 AM UTC, comment #3: 

Confirmed using Doug's simplified example from comment #2.

As a simple workaround, if you know that the numbers are in decimal format then you can use either %u for unsigned or %d for optionally signed integers.  The following format works

frm = '"%d" ,"%i","%i","%i","%i","%i"\n';
sscanf ('"08"', frm)
ans =  8

The trouble seems to be that %i must recognize decimal, octal, and hexadecimal.  In this case, the first character of the number is a '0' which seems to trigger the octal conversion format.  But there is no valid octal number for 8 and 9 so it returns '0' for that conversion.

On the other hand, anything you do which breaks the recognition of the number as octal will cause it to work.  For example,

sscanf ('"+08"', frm)
ans =  8
sscanf ('"-08"', frm)
ans = -8

It would be interesting to know if Matlab gets this right.  If anyone has access, could they run the following two tests?

sscanf ('"09"', '"%i"')
sscanf ('"079"', '"%i"')

Rik <rik5>
Project Administrator
Sat 21 Nov 2015 11:49:28 PM UTC, comment #2: 

more info

format short g
good  = '"01","02","03","04","05","06"';
good1  = '"07"';
bad1  = '"08"';
bad2 = '"09"';
good2  = '"10"';
good3 = '"8"';
good4 = '"9"';
frm = '"%i" ,"%i","%i","%i","%i","%i"\n';
v = sscanf(good,frm)'
v1 = sscanf(good1,frm)'
v2 = sscanf(bad1,frm)'
v3 = sscanf(bad2,frm)'
v4 = sscanf(good2,frm)'
v5 = sscanf(good3,frm)'
v6 = sscanf(good4,frm)'

From this we can see that 08 and 09 are wrong but 8 and 9 are ok
and 01 02 03 04 05 06 07 are ok also

Doug Stewart <dastew>
Sat 21 Nov 2015 06:59:37 PM UTC, comment #1: 

Here is a shorter version that shows the problem.

format short g
good  = '"11/8/2015 6:07:24 PM","46","46","48"';
bad1  = '"11/7/2015 6:08:24 PM","46","46","48"';
bad2  = '"11/7/2015 6:09:24 PM","46","46","48"';
good3  = '"11/7/2015 6:10:24 PM","46","46","48"';
frm = '"%i/%i/%i %i:%i:%i %2s" , "%i" , "%i"\n';
v = sscanf(good,frm)'
v1 = sscanf(bad1,frm)'
v2 = sscanf(bad2,frm)'
v3 = sscanf(good3,frm)'

Doug Stewart <dastew>
Sat 21 Nov 2015 06:02:43 PM UTC, original submission:  

While developing a script to read and plot weather station data from a CSV file using fscanf, I got an unexpected result.  After some analysis I tracked it to fscanf reading a
particular reacord.  It didn't seem to be a error in the data record and my orignial thought was an issue with the size of the data (this occurred on the 4324th record in the
CSV file).  Then I tried reading in the data as bytes converting to characters and using sscanf, but got the same result.  Then I cleared memory and executed the following
standalone script which replicated the issue without the concern for memory.  The problem is that fscanf did not read all of the "bad" record data.

format short g
good = '"11/7/2015 6:24:00 PM","46","46","48","69.2","99","46.2","28.82126","13.5","11/8/2015 11:03:00 AM","6.835083","42","7.456454","270","9.320568"';
bad = '"11/7/2015 6:36:00 PM","47","48","48","69.6","96","47.8","28.82126","13.47","11/7/2015 5:08:00 PM","2.485485","48","1.864114","270","6.213712"';
frm = '"%i/%i/%i %i:%i:%i %2s" , "%i" , "%i" , "%i" , "%f" , "%i" , "%f" ,"%f" , "%f" , "%i/%i/%i %i:%i:%i %2s" , "%f" , "%i" , "%f" , "%f", "%f"\n';
v = sscanf(good,frm)'

v =

 Columns 1 through 14:

          11           7        2015           6          24           0          80          77          46          46          48        69.2          99        46.2

 Columns 15 through 28:

      28.821        13.5          11           8        2015          11           3           0          65          77      6.8351          42      7.4565         270

 Column 29:



ans =

           1          29

v = sscanf(bad,frm)'

v =

 Columns 1 through 14:

          11           7        2015           6          36           0          80          77          47          48          48        69.6          96        47.8

 Columns 15 through 21:

      28.821       13.47          11           7        2015           5           0

>> size(v)

ans =

           1          21




(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:

No files currently attached


Depends on the following items: None found

Items that depend on this one: None found


Carbon-Copy List
  • -email is unavailable- added by philipnienhuis
  • -email is unavailable- added by lachlan (Posted a comment)
  • -email is unavailable- added by ansi512 (Posted a comment)
  • -email is unavailable- added by suchland (Posted a comment)
  • -email is unavailable- added by rik5 (Posted a comment)
  • -email is unavailable- added by dastew (Posted a comment)
  • -email is unavailable- added by None (Submitted the item)

    Do you think this task is very important?
    If so, you can add your encouragement to it.
    This task has 0 encouragements so far.

    Only project members can vote.




    Follow 17 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2019-06-08 mtmiller Carbon-CopyRemoved 80942 => -
    2019-06-08 rik5 StatusPostponed => Fixed
        Open/ClosedOpen => Closed
    2016-11-23 rik5 Severity3 - Normal => 2 - Minor
    2016-04-21 philipnienhuis Carbon-Copy- => Added philipnienhuis
    2016-04-20 mtmiller Priority5 - Normal => 3 - Low
        StatusNone => Postponed
        SummaryIncorrect result from fscanf and scanf with %i format => scanf with %i and out of range octal does not set errmsg
    2015-12-07 philipnienhuis Item GroupIncorrect Result => Matlab Compatibility
        StatusInvalid => None
        Open/ClosedClosed => Open
        Release4.0.0 => dev
    2015-11-24 mtmiller StatusConfirmed => Invalid
        Open/ClosedOpen => Closed
    2015-11-23 rik5 StatusNone => Confirmed
        Operating SystemMicrosoft Windows => Any
        SummaryIncorrect result from fscanf and scanf => Incorrect result from fscanf and scanf with %i format

    Back to the top

    Powered by Savane 3.5