bugGNU Octave - Bugs: bug #46493, scanf with %i and out of range...

 
 

bug #46493: scanf with %i and out of range octal does not set errmsg

Submitter:  None
Submitted:  Sat 21 Nov 2015 06:02:43 PM UTC
   
 
Category:  Octave Function Severity:  2 - Minor
Priority:  3 - Low Item Group:  Matlab Compatibility
Status:  Fixed Assigned to:  None
Originator Name:  Richard Suchland Originator Email:  -email is unavailable-
Open/Closed:  * Closed Release:  * dev
Operating System:  * Any Fixed Release:  None
Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Sat 08 Jun 2019 03:16:21 AM UTC, comment #14: 

This has been fixed by this changeset


changeset:   27137:30f53f7a7293
user:        Rik <rik@octave.org>
date:        Mon Jun 03 09:04:41 2019 -0700
summary:     Return correct error messages on octave-streams when scanf fails (bug #56396)


Closing report.


Rik <rik5>
Group administrator
Mon 01 Feb 2016 10:14:59 AM UTC, comment #13: 

Rik, I didn't notice that those variables were private until I tried writing a patch...

My understanding is that it is currently not possible to have a warning that sets the state without issuing a message, which is what is really wanted here.  I think the best solution is to address bug #41028 ("warning off" should still set state, for Matlab compatibility) by having three states for each warning: "on" (message and set state), "off" (set state, but don't message), "disabled" or similar (neither message nor set state).  Then we could have a warning for this that is "off" by default.

Lachlan Andrew <lachlan>
Sun 31 Jan 2016 04:39:33 AM UTC, comment #12: 

The issue is that Vlast_warning_id and Vlast_warning_message are meant to be private variables only modified through explicit calls to error and warning.  If we have functions changing internal state through backdoor mechanisms then the code can become unstable.  Worse, the resulting problems can be very hard to debug.

On the other hand, maybe it would be okay to call warning and then check, as you suggest, for both a warning or os.error being set.

Rik <rik5>
Group administrator
Fri 29 Jan 2016 11:36:02 PM UTC, comment #11: 

How about setting  Vlast_warning_id and Vlast_warning_message, and checking them as well as os.error() in  Ffscanf?

Depending on the resolution of bug #41028, the cleanest thing would just be to have a warning that defaults to "off, but set state", but until that is resolved, it would be possible just to set Vlast_warning_id and Vlast_warning_message directly.

Lachlan Andrew <lachlan>
Wed 20 Jan 2016 02:59:28 AM UTC, comment #10: 

I found the point in the code that needs to be modified.  It is in corefcn/oct-stream.cc.  I added the following at line 1155:


else if (c2 == '8' || c2 == '9')
{
  // FIXME: Would like to set error state on octave stream.
  // See bug #46493.  But only std::istream is input to fcn
  // error ("internal failure to match octal format");
  ref = 0;
}


Unfortunately, there is no easy way to get at the error function of the octave-stream which in order to set the errmsg variable.  So, this probably won't get fixed.

Rik <rik5>
Group administrator
Mon 07 Dec 2015 06:52:16 PM UTC, comment #9: 

Reopening this report after a more thorough check with Matlab.

Matlab r2014a (empty lines removed):

>> [VAL, COUNT, ERRMSG] = sscanf('''09''', '''%i''')
VAL =
     0
COUNT =
     1
ERRMSG =
Matching failure in format.
>> [VAL, COUNT, ERRMSG] = sscanf('"079"', '"%i"')
VAL =
     7
COUNT =
     1
ERRMSG =
Matching failure in format.
>>


whereas Octave-4.1.0+ gives:

>> [VAL, COUNT, ERRMSG] = sscanf('''09''', '''%i''')
VAL = 0
COUNT =  1
ERRMSG =
>> [VAL, COUNT, ERRMSG] = sscanf('"079"', '"%i"')
VAL =  7
COUNT =  1
ERRMSG =
>>


So Matlab, when explicitly queried using the ERRMSG output arg, informs one that it found illegal characters, while Octave just remains silent.

Release    => dev
Item group => Matlab Compatibility


Philip Nienhuis <philipnienhuis>
Group Member
Wed 25 Nov 2015 03:15:44 PM UTC, comment #8: 

I think that follows the conventions of the standard C scanf function as well.

Regardless, solution for OP found, Octave working as expected, moving on :)

Mike Miller <mtmiller>
Group Member
Wed 25 Nov 2015 02:07:07 PM UTC, comment #7: 

@Mike:
Whether it's a bug or not might be debatable.
All we can say is that Octave is compatible with Matlab (but perhaps bug-for-bug compatible).
IMO it can be argued that "09" should also be something that "breaks the recognition of the number as octal" (as per comment #3).

But Matlab itself has the final word:
In table Input Arguments of http://nl.mathworks.com/help/matlab/ref/sscanf.html: "... If initial digit is 0, it is base 8."

I'd conclude that Matlab's base detection logic is a bit quick and dirty, but we have to live with it.

Philip Nienhuis <philipnienhuis>
Group Member
Tue 24 Nov 2015 04:06:30 PM UTC, comment #6: 

Based on comment #5, it looks like Octave has the correct behavior, so no bug here.

Mike Miller <mtmiller>
Group Member
Tue 24 Nov 2015 01:29:35 PM UTC, comment #5: 

Just stumble over this bugreport, here matlabs result from Rik's suggested test of comment #2:

sscanf('"09"', '"%i"')
ans =
     0
sscanf('"079"', '"%i"')
ans =
     7

Anselm Köhler <ansi512>
Mon 23 Nov 2015 01:51:27 AM UTC, comment #4: 

I replaced all occurrences of %i %d per the suggested workaround and using it resulted in correct reading of all data in my dataset!
Thanks

Richard Suchland <suchland>
Mon 23 Nov 2015 12:49:05 AM UTC, comment #3: 

Confirmed using Doug's simplified example from comment #2.

As a simple workaround, if you know that the numbers are in decimal format then you can use either %u for unsigned or %d for optionally signed integers.  The following format works


frm = '"%d" ,"%i","%i","%i","%i","%i"\n';
sscanf ('"08"', frm)
ans =  8


The trouble seems to be that %i must recognize decimal, octal, and hexadecimal.  In this case, the first character of the number is a '0' which seems to trigger the octal conversion format.  But there is no valid octal number for 8 and 9 so it returns '0' for that conversion.

On the other hand, anything you do which breaks the recognition of the number as octal will cause it to work.  For example,


sscanf ('"+08"', frm)
ans =  8
sscanf ('"-08"', frm)
ans = -8


It would be interesting to know if Matlab gets this right.  If anyone has access, could they run the following two tests?


sscanf ('"09"', '"%i"')
sscanf ('"079"', '"%i"')



Rik <rik5>
Group administrator
Sat 21 Nov 2015 11:49:28 PM UTC, comment #2: 

more info


format short g
good  = '"01","02","03","04","05","06"';
good1  = '"07"';
bad1  = '"08"';
bad2 = '"09"';
good2  = '"10"';
good3 = '"8"';
good4 = '"9"';
frm = '"%i" ,"%i","%i","%i","%i","%i"\n';
v = sscanf(good,frm)'
v1 = sscanf(good1,frm)'
v2 = sscanf(bad1,frm)'
v3 = sscanf(bad2,frm)'
v4 = sscanf(good2,frm)'
v5 = sscanf(good3,frm)'
v6 = sscanf(good4,frm)'


From this we can see that 08 and 09 are wrong but 8 and 9 are ok
and 01 02 03 04 05 06 07 are ok also

Doug Stewart <dastew>
Sat 21 Nov 2015 06:59:37 PM UTC, comment #1: 

Here is a shorter version that shows the problem.


format short g
good  = '"11/8/2015 6:07:24 PM","46","46","48"';
bad1  = '"11/7/2015 6:08:24 PM","46","46","48"';
bad2  = '"11/7/2015 6:09:24 PM","46","46","48"';
good3  = '"11/7/2015 6:10:24 PM","46","46","48"';
frm = '"%i/%i/%i %i:%i:%i %2s" , "%i" , "%i"\n';
v = sscanf(good,frm)'
v1 = sscanf(bad1,frm)'
v2 = sscanf(bad2,frm)'
v3 = sscanf(good3,frm)'


Doug Stewart <dastew>
Sat 21 Nov 2015 06:02:43 PM UTC, original submission:  

While developing a script to read and plot weather station data from a CSV file using fscanf, I got an unexpected result.  After some analysis I tracked it to fscanf reading a
particular reacord.  It didn't seem to be a error in the data record and my orignial thought was an issue with the size of the data (this occurred on the 4324th record in the
CSV file).  Then I tried reading in the data as bytes converting to characters and using sscanf, but got the same result.  Then I cleared memory and executed the following
standalone script which replicated the issue without the concern for memory.  The problem is that fscanf did not read all of the "bad" record data.


format short g
good = '"11/7/2015 6:24:00 PM","46","46","48","69.2","99","46.2","28.82126","13.5","11/8/2015 11:03:00 AM","6.835083","42","7.456454","270","9.320568"';
bad = '"11/7/2015 6:36:00 PM","47","48","48","69.6","96","47.8","28.82126","13.47","11/7/2015 5:08:00 PM","2.485485","48","1.864114","270","6.213712"';
frm = '"%i/%i/%i %i:%i:%i %2s" , "%i" , "%i" , "%i" , "%f" , "%i" , "%f" ,"%f" , "%f" , "%i/%i/%i %i:%i:%i %2s" , "%f" , "%i" , "%f" , "%f", "%f"\n';
v = sscanf(good,frm)'

v =

 Columns 1 through 14:

          11           7        2015           6          24           0          80          77          46          46          48        69.2          99        46.2

 Columns 15 through 28:

      28.821        13.5          11           8        2015          11           3           0          65          77      6.8351          42      7.4565         270

 Column 29:

      9.3206

size(v)

ans =

           1          29


v = sscanf(bad,frm)'

v =

 Columns 1 through 14:

          11           7        2015           6          36           0          80          77          47          48          48        69.6          96        47.8

 Columns 15 through 21:

      28.821       13.47          11           7        2015           5           0

>> size(v)

ans =

           1          21

>>


Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by philipnienhuis
  • -email is unavailable- added by lachlan (Posted a comment)
  • -email is unavailable- added by ansi512 (Posted a comment)
  • -email is unavailable- added by suchland (Posted a comment)
  • -email is unavailable- added by rik5 (Posted a comment)
  • -email is unavailable- added by dastew (Posted a comment)
  • -email is unavailable- added by None (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 17 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2019-06-08 mtmiller Carbon-CopyRemoved 80942 -
    2019-06-08 rik5 StatusPostponed Fixed
        Open/ClosedOpen Closed
    2016-11-23 rik5 Severity3 - Normal 2 - Minor
    2016-04-21 philipnienhuis Carbon-Copy- Added philipnienhuis
    2016-04-20 mtmiller Priority5 - Normal 3 - Low
        StatusNone Postponed
        SummaryIncorrect result from fscanf and scanf with %i format scanf with %i and out of range octal does not set errmsg
    2015-12-07 philipnienhuis Item GroupIncorrect Result Matlab Compatibility
        StatusInvalid / Not an Octave Bug None
        Open/ClosedClosed Open
        Release4.0.0 dev
    2015-11-24 mtmiller StatusConfirmed Invalid / Not an Octave Bug
        Open/ClosedOpen Closed
    2015-11-23 rik5 StatusNone Confirmed
        Operating SystemMicrosoft Windows Any
        SummaryIncorrect result from fscanf and scanf Incorrect result from fscanf and scanf with %i format

    Back to the top

    Powered by Savane 3.13-f8d8.
    Corresponding source code