buggrep - Bugs: bug #19637, \w and [[:alnum:]] not equivalent...

 
 

You are not allowed to post comments on this tracker with your current authentication level.

bug #19637: \w and [[:alnum:]] not equivalent in multibyte locale

Submitted by:  None
Submitted on:  Fri 20 Apr 2007 08:26:28 AM UTC  
 
Category:  None Severity:  3 - Normal
Item Group:  None Status:  None
Privacy:  Public Assigned to:  None
Open/Closed:  Open

Fri 20 Apr 2007 12:14:25 PM UTC, comment #1: 

Confirmed.  So this is two bugs:

1) The man page should not say that \w equals [[:alnum:]], as the first includes also the underscore.
2) \w does not match accented characters in an utf8 locale.

$ export LC_ALL=nl_NL
$ echo -e " ee\n ëë\n __\n" | src/grep -E '\w'
 ee
 ëë
 __
$ echo -e " ee\n ëë\n __\n" | src/grep -E '[[:alnum:]]'
 ee
 ëë

[switch the Konsole's encoding from iso-8859-1 to utf8 and retype the lines instead of recalling history]

$ export LC_ALL=nl_NL.utf8
$ echo -e " ee\n ëë\n __\n" | src/grep -E '\w'
 ee
 __
$ echo -e " ee\n ëë\n __\n" | src/grep -E '[[:alnum:]]'
 ee
 ëë

Benno Schulenberg <bens>
Fri 20 Apr 2007 08:26:28 AM UTC, original submission:  

In grep's manpage I found:

  The symbol \w is a synonym for [[:alnum:]]

This is true in a single-byte locale
(terminal = xterm -en iso-8859-1):

 echo -e "a\nä\n" | LC_ALL=de_DE ./grep -E '\w'
 a
 ä
 echo -e "a\nä\n_" | LC_ALL=de_DE ./grep -E '[[:alnum:]]'
 a
 ä

But not in an utf-8 locale
(terminal xterm -u8):

 echo -e "a\nä" | LC_ALL=de_DE.utf8 ./grep -P '\w'
 a
 echo -e "a\nä\n_" | LC_ALL=de_DE.utf8 ./grep -E '[[:alnum:]]'
 a
 ä

System:

  openSUSE 10.2

  grep from CVS

[volga:src] uname -a
Linux volga 2.6.18.8-0.1-default #1 SMP Fri Mar 2 13:51:59 UTC 2007 i686 i686 i386 GNU/Linux
[volga:src] locale -V
locale (GNU libc) 2.5
...
[volga:src] ./grep -V
GNU grep 2.5.1-cvs
...

Thanks, Sebastian

(wastl[]cis.uni-muenchen.de)

 
Anonymous

 

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by bonzini (Updated the item)
  • -email is unavailable- added by bens (Posted a comment)
  •  

    Do you think this task is very important?
    If so, you can add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

     

     

    Follow 4 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2010-02-06 bonzini Summary\\w and [[:alnum:]] not equivalent in multibyte locale => \w and [[:alnum:]] not equivalent in multibyte locale
    2007-04-20 bens Carbon-Copy- => Added wastl@cis.uni-muenchen.de
    2007-04-20 bens Carbon-CopyRemoved -email is unavailable- => -
    2007-04-20 bens Carbon-Copy- => Added wastl@cis.uni-muenchen.de

    Back to the top


    Powered by Savane 3.5