bugGNU Octave - Bugs: bug #54592, regexp should escape unpaired...

 
 

bug #54592: regexp should escape unpaired brackets and parentheses for Matlab compatibility

Submitter:  Docf <docfr>
Submitted:  Wed 29 Aug 2018 12:39:44 PM UTC
   
 
Category:  Octave Function Severity:  3 - Normal
Priority:  4 Item Group:  Matlab Compatibility
Status:  Confirmed Assigned to:  None
Originator Name:  Open/Closed:  * Open
Release:  * dev Operating System:  * Any
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Mon 03 Sep 2018 05:12:19 PM UTC, comment #11: 

So it would seem that Matlab pre-processes the regular expression and escapes non-paired special characters such as '[', ']', '(', ')'.  It also seems to be smart enough to recognize when one of these characters has already been escaped.

Marking as confirmed.  The easy workaround, for the time being, is to escape unpaire brackets and parentheses yourself in the regular expression.  Because there is a workaround I am lowering the priority slightly to 4.

Rik <rik5>
Group administrator
Mon 03 Sep 2018 07:45:31 AM UTC, comment #10: 


>> version
ans =
    '9.5.0.882065 (R2018b) Prerelease'
           %%% Comment 7
>> regexprep ('tes(t', '(', '')
ans =
    'test'
>> regexprep ('tes[t', '[', '')
ans =
    'test'


Philip Nienhuis <philipnienhuis>
Group Member
Mon 03 Sep 2018 06:36:34 AM UTC, comment #9: 


>> version
ans = '9.4.0.813654 (R2018a)'

>> regexprep('tes)t', '(', '')
ans = 'tes)t'

>> regexprep('tes]t', '[', '')
ans = 'tes]t'

>> regexprep('tes)t', '())', '#')
ans ='tes#t'

>> regexprep('tes)t', '(\))', '#')
ans = 'tes#t'


A.R. Burgers <arb>
Fri 31 Aug 2018 10:03:03 PM UTC, comment #8: 

I asked for the opening parenthesis test in the first batch of regexprep tests and the results are that it works without error.


regexprep ('tes)t', '(', '')
ans =
    'tes)t'


Someone should perform the open bracket '[' test, but it seems likely that it too will pass.

If it is escaping unmatched meta-characters, does it recognize escapes?


regexprep ('tes)t', '())', '#')
regexprep ('tes)t', '(\))', '#')



Rik <rik5>
Group administrator
Fri 31 Aug 2018 05:07:21 PM UTC, comment #7: 

Can someone test the same, with an unmatched opening bracket? For example


regexprep ('tes(t', '(', '')
regexprep ('tes[t', '[', '')


It looks to me like both Perl and sed treat an unmatched ']' the same as '\]', but an unmatched '[' is an error. This is specifically documented in the Perl RE syntax as an exception. The ']' character is called a "sometimes metacharacter", but the '()' characters are always metacharacters.

However, POSIX extended regular expression syntax seems to allow a ')' to be interpreted as a normal character when there is no corresponding '('. So under POSIX ERE rules, ')' is also a "sometimes metacharacter", but not under Perl RE rules.

This can be confimed with grep


$ echo 'tes)t' | grep -E ')'
tes)t
$ echo 'tes)t' | grep -P ')'
grep: unmatched parentheses


Mike Miller <mtmiller>
Group Member
Fri 31 Aug 2018 09:12:42 AM UTC, comment #6: 


computer
ans =
    'PCWIN64'
version
ans =
    '9.5.0.882065 (R2018b) Prerelease'
regexprep ('tes)t', ')', '')
ans =
    'test'
regexprep ('tes)t', ')', 'abc')
ans =
    'tesabct'
regexprep ('tes)t', '(', '')
ans =
    'tes)t'
regexprep ('te(s)t', '(s)', '')
ans =
    'te()t'


and e.g.,:

>> regexprep ('tes]t', ']', 'abc')
ans =
    'tesabct'
>> regexprep ('tes[]t', '[]', 'abc')
ans =
    'tes[]t'


Indeed, as the OP wrote Matlab apparently searches for matching parens / sq. brackets before deciding they're to be treated as special characters.

Philip Nienhuis <philipnienhuis>
Group Member
Fri 31 Aug 2018 05:43:08 AM UTC, comment #5: 

Thank you for your support. The diary file is attached.
I hope it will help you.

(file #44907)

Docf <docfr>
Thu 30 Aug 2018 08:52:57 PM UTC, comment #4: 

I'm very surprised this works.  Using Perl directly


perl -pe 's/)//'  "tes)t"
Unmatched ) in regex; marked by <-- HERE in m/) <-- HERE / at -e line 1.


Could you run the following in your version of Matlab and then upload the diary file?


diary on
computer
version
regexprep ('tes)t', ')', '')
regexprep ('tes)t', ')', 'abc')
regexprep ('tes)t', '(', '')
regexprep ('te(s)t', '(s)', '')
diary off



Rik <rik5>
Group administrator
Thu 30 Aug 2018 08:02:43 AM UTC, comment #3: 

Ok, so I'll reopen the report and mark it as ML compatibility.

Pantxo Diribarne <pantxo>
Group Member
Wed 29 Aug 2018 04:13:31 PM UTC, comment #2: 

Thank you for the quick reply and your great support!

In Matlab the given problem line works fine, without escape characters.

As I understood, any deviation between Octave and Matlab in behavior is considered as bug.

Thank you!

Docf <docfr>
Wed 29 Aug 2018 02:23:00 PM UTC, comment #1: 

Parenthesis are reserved grouping characters (see "help regexp") so you need to escape them:


regexprep("tes)t", '\)', '')


Note the \ character.
Closing report as invalid.

Pantxo Diribarne <pantxo>
Group Member
Wed 29 Aug 2018 12:39:44 PM UTC, original submission:  

When parentheses  should be processed e.g. regexprep("tes)t", ')', '') Octave returns "error: regexprep: unmatched parentheses at position 0 of expression"

The command regexprep("tes!t", '!', '') will just remove the !
This works as expected. There seems to be check for balance of parentheses this must be removed here.

Docf <docfr>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #44907:  diary added by docfr (331B - application/octet-stream - matlab diary)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by arb (Posted a comment)
  • -email is unavailable- added by mtmiller (Posted a comment)
  • -email is unavailable- added by rik5 (Posted a comment)
  • -email is unavailable- added by pantxo (Posted a comment)
  • -email is unavailable- added by docfr (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 12 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2018-09-03 rik5 Priority5 - Normal 4
        StatusNeed Info Confirmed
        Release4.4.0 dev
        Summaryregexprep gets confused when parentheses should be processed regexp should escape unpaired brackets and parentheses for Matlab compatibility
    2018-08-31 docfr Attached File- Added diary, #44907
    2018-08-30 rik5 StatusConfirmed Need Info
    2018-08-30 pantxo Item GroupUnexpected Error or Warning Matlab Compatibility
        StatusInvalid / Not an Octave Bug Confirmed
        Open/ClosedClosed Open
        Operating SystemMicrosoft Windows Any
    2018-08-29 pantxo StatusNone Invalid / Not an Octave Bug
        Open/ClosedOpen Closed

    Back to the top

    Powered by Savane 3.13-4448.
    Corresponding source code