buggrep - Bugs: bug #19637, \w and [[:alnum:]] not equivalent...

 
 

You are not allowed to post comments on this tracker with your current authentification level.

bug #19637: \w and [[:alnum:]] not equivalent in multibyte locale

Submitted by:  None
Submitted on:  Fri 20 Apr 2007 08:26:28 AM UTC  
 
Category: NoneSeverity: 3 - Normal
Item Group: NoneStatus: None
Privacy: PublicAssigned to: None
Open/Closed: Open

Fri 20 Apr 2007 12:14:25 PM UTC, comment #1:

Confirmed. So this is two bugs:

1) The man page should not say that \w equals [[:alnum:]], as the first includes also the underscore.
2) \w does not match accented characters in an utf8 locale.

$ export LC_ALL=nl_NL
$ echo -e " ee\n ëë\n __\n" | src/grep -E '\w'
ee
ëë
__
$ echo -e " ee\n ëë\n __\n" | src/grep -E '[[:alnum:]]'
ee
ëë

[switch the Konsole's encoding from iso-8859-1 to utf8 and retype the lines instead of recalling history]

$ export LC_ALL=nl_NL.utf8
$ echo -e " ee\n ëë\n __\n" | src/grep -E '\w'
ee
__
$ echo -e " ee\n ëë\n __\n" | src/grep -E '[[:alnum:]]'
ee
ëë

Benno Schulenberg <bens>
Fri 20 Apr 2007 08:26:28 AM UTC, original submission:

In grep's manpage I found:

The symbol \w is a synonym for [[:alnum:]]

This is true in a single-byte locale
(terminal = xterm -en iso-8859-1):

echo -e "a\nä\n" | LC_ALL=de_DE ./grep -E '\w'
a
ä
echo -e "a\nä\n_" | LC_ALL=de_DE ./grep -E '[[:alnum:]]'
a
ä

But not in an utf-8 locale
(terminal xterm -u8):

echo -e "a\nä" | LC_ALL=de_DE.utf8 ./grep -P '\w'
a
echo -e "a\nä\n_" | LC_ALL=de_DE.utf8 ./grep -E '[[:alnum:]]'
a
ä

System:

openSUSE 10.2

grep from CVS

[volga:src] uname -a
Linux volga 2.6.18.8-0.1-default #1 SMP Fri Mar 2 13:51:59 UTC 2007 i686 i686 i386 GNU/Linux
[volga:src] locale -V
locale (GNU libc) 2.5
...
[volga:src] ./grep -V
GNU grep 2.5.1-cvs
...

Thanks, Sebastian

(wastl[]cis.uni-muenchen.de)

Anonymous

 

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by bonzini (Updated the item)
  • -unavailable- added by bens (Posted a comment)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 4 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Sat 06 Feb 2010 01:13:45 PM UTCbonziniSummary\\w and [[:alnum:]] not equivalent in multibyte locale=>\w and [[:alnum:]] not equivalent in multibyte locale
    Fri 20 Apr 2007 12:16:58 PM UTCbensCarbon-Copy-=>Added -unavailable-
    Fri 20 Apr 2007 12:15:41 PM UTCbensCarbon-CopyRemoved -unavailable-=>-
    Fri 20 Apr 2007 12:14:25 PM UTCbensCarbon-Copy-=>Added -unavailable-

    Back to the top


    Powered by Savane 3.1-cleanup