bugGNU Octave - Bugs: bug #35683, regexp crash (named subexp/nested...

 
 

bug #35683: regexp crash (named subexp/nested paren; maybe related to #29438)

Submitter:  None
Submitted:  Thu 01 Mar 2012 07:48:05 PM UTC
   
 
Category:  Libraries Severity:  4 - Important
Priority:  5 - Normal Item Group:  Segfault, Bus Error, etc.
Status:  Fixed Assigned to:  None
Originator Name:  Philipp Kutin Originator Email:  -email is unavailable-
Open/Closed:  * Closed Release:  * dev
Operating System:  * GNU/Linux Fixed Release:  None
Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Mon 12 Mar 2012 05:42:50 PM UTC, comment #5: 

New HG tip + ?: hints work great on my data, thanks!

Philipp Kutin <pkutin>
Sun 11 Mar 2012 08:46:18 PM UTC, comment #4: 

I've fixed the issue and committed the patch on the stable branch (http://hg.savannah.gnu.org/hgweb/octave/rev/721be41ea988) of Octave which will become 3.6.2 in the near future.

You can either build from Mercurial sources, wait for the 3.6.2 release, or change your regexp pattern to work around this issue.


Rik <rik5>
Group administrator
Sun 11 Mar 2012 06:03:04 PM UTC, comment #3: 

Interestingly, my system does not behave the same as yours and I don't see segfaults for your reduced testcases.  This is probably due to subtle differences in the way things are laid out in memory.  I'm sure the memory corruption is occurring, but it doesn't always lead to an actual segfault.

The following string and pattern combination always segfaults for me (tested on versions 3.2.4, 3.4.0, 3.4.3, 3.6.1, and the dev).


str = 'char short xyz';
ptn = '(?<label>(char|short)\s+)';
regexp (str, ptn, 'names')


The problem seems to be with the mixing of named and unnamed match buffers.  The outer parentheses in the pattern above create a match buffer, which happens to be named label.  The inner parenthesis, which I know you are intending to use only for grouping, also creates an unnamed match buffer.  When you ask for the named match tokens Octave indexes into an array of names.  It finds the first name, <label>, but when it goes to find the second name it oversteps the array bounds.

A solution in this case is to use alternation without creating a back reference.  The syntax for this is '(?:A|B)'.  While the example above segfaults, this one works.


str = 'char short xyz';
ptn = '(?<label>(?:char|short)\s+)';
regexp (str, ptn, 'names')


Another possibility is to explicitly name every capture backreference.  I tried that with your original pattern and it no longer segfaults.  (I don't think it's giving you what you want but that is a different question about what the regexp should be picking out from the string.)

The pattern that doesn't segfault is


pattern = '(?<typestr>(?<unname1>(?<unname2>char|short|int|long|signed|unsigned|float|double|int8|uint8|int16|uint16|int32|uint32|int64|uint64|int8_t|uint8_t|int16_t|uint16_t|int32_t|uint32_t|int64_t|uint64_t|BYTE|UBYTE|WORD|DWORD|QWORD)\s+)+)(?<name>[a-zA-Z_][a-zA-Z0-9_]*)\s*(?<length>\[[0-9]+\])?\s*;'


Where I have used 'unname1' and 'unname2' for the previously implicit capture buffers.

Obviously, there is still a deeper problem that bad input, no matter how malformed, shouldn't cause the program to segfault.  It should produce a warning or error message and return to the prompt.

Rik <rik5>
Group administrator
Fri 09 Mar 2012 05:40:18 PM UTC, comment #2: 

Here are some reduced testcases:

It's still dependent on the string to be matched:

octave:1> simplepat = '(?<typestr>((char|short)\s+)+)';
octave:2> s = regexp('', simplepat, 'names');
octave:3> s = regexp('unsigned', simplepat, 'names');
octave:4> s = regexp('unsigned char Impedance;', simplepat, 'names');
panic: Segmentation fault -- stopping myself...


However,

octave:1> simplepat = '(?<typestr>(char\s+)+)';
octave:2> s = regexp('unsigned char Impedance;', simplepat, 'names');
# no crash

So it seems to be related to the alternative operator inside the inner group.

Finally,

octave:1> simplepat = '(?<typestr>((char|short)\s+)+)';
octave:2> s = regexp('unsigned char Impedance;', simplepat)  # no 3rd 'names' arg
panic: Segmentation fault -- stopping myself...


and


octave:1> simplepat = '(char|short)+';
octave:2> s = regexp('unsigned char Impedance;', simplepat)
s =  10
octave:3> simplepat = '((char|short)+)+';
octave:4> s = regexp('unsigned char Impedance;', simplepat)
s =  10
octave:5> simplepat = '((char|short)\s+)+';
octave:6> s = regexp('unsigned char Impedance;', simplepat)
s =  10
octave:7> simplepat = '(((char|short)+)+)';
octave:8> s = regexp('unsigned char Impedance;', simplepat)
s =  10

So it also seems to have something to do with the named subexpression mechanism.

Philipp Kutin <pkutin>
Fri 02 Mar 2012 05:54:06 PM UTC, comment #1: 

I can confirm that this causes a segfault on a recent tip (3/2/12).  It's too bad there's not a slightly simpler example.  On the other hand, you're right to say the code shouldn't crash even when the pattern is faulty.

Rik <rik5>
Group administrator
Thu 01 Mar 2012 07:48:05 PM UTC, original submission:  

The following produces a crash, after loading the attached data:


s = regexp(q(1).contents, pattern, 'names');


The pattern is well-formed as far as I can see:


(?<typestr>((char|short|int|long|signed|unsigned|float|double|int8|uint8|int16|uint16|int32|uint32|int64|uint64|int8_t|uint8_t|int16_t|uint16_t|int32_t|uint32_t|int64_t|uint64_t|BYTE|UBYTE|WORD|DWORD|QWORD)\s+)+)(?<name>[a-zA-Z_][a-zA-Z0-9_]*)\s*(?<length>\[[0-9]+\])?\s*;


(of course, invalid regexps should not cause a crash either.)

I'm using the tip of the 'stable' branch on GNU/Linux and x86.

The crash is dependent on the string to be matched, e.g.

s = regexp('qwe int asd', pattern, 'names');

works fine.

When called without output args,

regexp(q(1).contents, pattern, 'names')

the following is printed in addition to the segfault message:


terminate called after throwing an instance of 'std::logic_error'
what(): basic_string::_S_construct null not valid



A backtrace using an optimized build is attached, too.

Finally, don't bash me for trying to parse a real language using regexps, I know it's braindead ;)

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #25232:  structq added by None (4KiB - application/octet-stream - structq and pattern are to be loaded with 'load'. Backtrace with optimized build.)
file #25233:  regexp_crash_1.log added by None (11KiB - text/x-log - structq and pattern are to be loaded with 'load'. Backtrace with optimized build.)
file #25234:  pattern added by None (409B - application/octet-stream - structq and pattern are to be loaded with 'load'. Backtrace with optimized build.)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by pkutin (Posted a comment)
  • -email is unavailable- added by rik5 (Posted a comment)
  • -email is unavailable- added by None (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 7 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2012-03-11 rik5 StatusConfirmed Fixed
        Open/ClosedOpen Closed
    2012-03-11 rik5 Severity3 - Normal 4 - Important
    2012-03-02 rik5 StatusNone Confirmed
    2012-03-01 None Attached File- Added structq, #25232
        Attached File- Added regexp_crash_1.log, #25233
        Attached File- Added pattern, #25234

    Back to the top

    Powered by Savane 3.13-02a9.
    Corresponding source code