bugGNU Octave - Bugs: bug #37092, backslash escape character in...

 
 

bug #37092: backslash escape character in regexp

Submitter:  Guillaume <gyom>
Submitted:  Tue 14 Aug 2012 12:08:02 PM UTC
   
 
Category:  Libraries Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Matlab Compatibility
Status:  Fixed Assigned to:  rik5
Originator Name:  Guillaume Open/Closed:  * Closed
Release:  * dev Operating System:  * Any
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Thu 18 Oct 2012 03:18:13 AM UTC, comment #10: 

I fixed this problem in this changeset (http://hg.savannah.gnu.org/hgweb/octave/rev/9db32cabeacf).  You will need to build from Mercurial sources with a version greater than or equal to 15541:9db32cabeacf in order to see the fix.

Rik <rik5>
Group administrator
Thu 23 Aug 2012 04:43:45 PM UTC, comment #9: 

On my local machine, with my patch applied, your latest test runs fine so I think the single patch will fix both problems.

Rik <rik5>
Group administrator
Thu 23 Aug 2012 04:02:54 PM UTC, comment #8: 

Thanks for the update Rik. If it helps, I can do some tests if you give me a patch.

Perhaps totally unrelated, but I noticed another problem with regexp:

str='$Id: file.m 1234 2012-08-23 09:12:27Z user $';
regexp(str,['\$Id: (?<file>\S+) (?<id>[0-9]+) (?<date>\S+) ' ...
            '(\S+Z) (?<author>\S+) \$'],'names','once')


MATLAB returns

      file: 'file.m'
        id: '1234'
      date: '2012-08-23'
    author: 'user'


as expected, while Octave returns empty fields.

Guillaume <gyom>
Wed 22 Aug 2012 09:02:01 PM UTC, comment #7: 

This small little behavior has exposed some real problems in the regexprep code.  I've had to rewrite a large amount of C++ code.  I've got a solution that works now, but I want to run some of it by the octave-maintainers list before committing the changes.

Rik <rik5>
Group administrator
Thu 16 Aug 2012 10:42:11 AM UTC, comment #6: 

Thanks for looking into this.
With MATLAB R2012a, I get:


>> regexprep('\n','\\$','backslash')
ans =
\n

>> regexprep('a\','\\$','backslash')
ans =
abackslash

>> regexprep('a', '(a)', 'b\$1')
ans =
b$1

>> regexprep('a', '(a)', 'b\\$1')
ans =
b\a

>> regexprep('a','a','%%')
ans =
%%

>> regexprep('a','a','\i')
ans =
i

>> regexprep('a','a','b''c''e')
ans =
b'c'e


Guillaume <gyom>
Thu 16 Aug 2012 12:18:07 AM UTC, comment #5: 

Okay, I've done some digging and Octave is not quite handling things correctly.  Can you run a few tests on Matlab to see how they interpret things?


regexprep('\n','\\$','backslash')
regexprep('a\','\\$','backslash')
regexprep('a', '(a)', 'b\$1')
regexprep('a', '(a)', 'b\\$1')
regexprep('a','a','%%')
regexprep('a','a','\i')
regexprep('a','a','b''c''e')



Rik <rik5>
Group administrator
Wed 15 Aug 2012 04:16:52 PM UTC, comment #4: 

I'm going to re-open this issue.  From your testing, it appears that Octave is interpreting escape character sequences twice which is not correct.  I am going to change the Item Group to "Matlab Compatibility" since that is what is really being chased here.

What I think is going on is the following:


regexprep ('\b', '\\b', 'backslash')

The pattern '\\b' is being searched for escape sequences.
The search finds '\\' and replaces it with '\'.
The pattern is now '\b'
I think the search then errors in thinking it has found another escape sequence.


I am building a debug version of Octave and will check this out with gdb.

Rik <rik5>
Group administrator
Wed 15 Aug 2012 11:21:21 AM UTC, comment #3: 

Thanks for your comments.

With MATLAB R2012a, I get this:

>> regexprep('\','\\','backslash')
ans =
backslash
>> regexprep('\b','\\b','backslash')
ans =
backslash


Let me know if you need other user cases.
I came across this with some MATLAB code I use to escape a string for insertion in LaTeX:

regexprep(string,{'\\','_'},{'\\\\','\\_'})

I am not sure how to write some code that will work similarly for both Octave and MATLAB.

Guillaume <gyom>
Tue 14 Aug 2012 10:26:13 PM UTC, comment #2: 

I added some extra documentation about this in this changeset (http://hg.savannah.gnu.org/hgweb/octave/rev/8593bed236f1).  Hopefully this helps with understanding why we have to do this now.

Rik <rik5>
Group administrator
Tue 14 Aug 2012 07:36:31 PM UTC, comment #1: 

This is the new behavior of regexp and regexprep in order to be Matlab-compatible.  Quoting from the NEWS file which lists significant changes from the last revision:


 ** For compatibility with Matlab, the regexp, regexpi, and regexprep
    functions now process backslash escapes in single-quoted pattern
    strings.  In addition, the regexprep function now processes backslash
    escapes in single-quoted replacement strings.


So, patterns like '\\' really look like "\\" which collapses to just "\" and this causes the problem.  It would be useful, if you have access to Matlab, to verify that they haven't done a special case for '\\'.  I rather doubt it though.

Either add an extra backslash when using escape sequences like "\n" or look at the regexptranslate function for another option.

Rik <rik5>
Group administrator
Tue 14 Aug 2012 12:08:02 PM UTC, original submission:  

Using the backslash escape character returns an error in regexprep and Octave dev with this call:

octave> regexprep('\','\\','backslash')
error: regexprep: \ at end of pattern at position 1 of expression


while it does not return the correct answer here:

octave> regexprep('\b','\\b','backslash')
ans = \b


It is fine with Octave 3.6.2.

Guillaume <gyom>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by rik5 (Posted a comment)
  • -email is unavailable- added by gyom (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 10 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2012-10-18 rik5 StatusIn Progress Fixed
        Open/ClosedOpen Closed
    2012-09-06 rik5 StatusConfirmed In Progress
        Assigned toNone rik5
    2012-08-15 rik5 Item GroupIncorrect Result Matlab Compatibility
        StatusNeed Info Confirmed
        Open/ClosedClosed Open
    2012-08-14 rik5 CategoryNone Libraries
        StatusNone Need Info
        Open/ClosedOpen Closed

    Back to the top

    Powered by Savane 3.13-4b48.
    Corresponding source code