buggrep - Bugs: bug #19637, \w and [[:alnum:]] not equivalent...


You are not allowed to post comments on this tracker with your current authentication level.

bug #19637: \w and [[:alnum:]] not equivalent in multibyte locale

Submitted by:  None
Submitted on:  Fri Apr 20 08:26:28 2007  
Category: NoneSeverity: 3 - Normal
Item Group: NoneStatus: None
Privacy: PublicAssigned to: None
Open/Closed: Open

Fri Apr 20 12:14:25 2007, comment #1:

Confirmed. So this is two bugs:

1) The man page should not say that \w equals [[:alnum:]], as the first includes also the underscore.
2) \w does not match accented characters in an utf8 locale.

$ export LC_ALL=nl_NL
$ echo -e " ee\n ëë\n __\n" | src/grep -E '\w'
$ echo -e " ee\n ëë\n __\n" | src/grep -E '[[:alnum:]]'

[switch the Konsole's encoding from iso-8859-1 to utf8 and retype the lines instead of recalling history]

$ export LC_ALL=nl_NL.utf8
$ echo -e " ee\n ëë\n __\n" | src/grep -E '\w'
$ echo -e " ee\n ëë\n __\n" | src/grep -E '[[:alnum:]]'

Benno Schulenberg <bens>
Fri Apr 20 08:26:28 2007, original submission:

In grep's manpage I found:

The symbol \w is a synonym for [[:alnum:]]

This is true in a single-byte locale
(terminal = xterm -en iso-8859-1):

echo -e "a\nä\n" | LC_ALL=de_DE ./grep -E '\w'
echo -e "a\nä\n_" | LC_ALL=de_DE ./grep -E '[[:alnum:]]'

But not in an utf-8 locale
(terminal xterm -u8):

echo -e "a\nä" | LC_ALL=de_DE.utf8 ./grep -P '\w'
echo -e "a\nä\n_" | LC_ALL=de_DE.utf8 ./grep -E '[[:alnum:]]'


openSUSE 10.2

grep from CVS

[volga:src] uname -a
Linux volga #1 SMP Fri Mar 2 13:51:59 UTC 2007 i686 i686 i386 GNU/Linux
[volga:src] locale -V
locale (GNU libc) 2.5
[volga:src] ./grep -V
GNU grep 2.5.1-cvs

Thanks, Sebastian




No files currently attached


Depends on the following items: None found

Items that depend on this one: None found


Carbon-Copy List
  • -unavailable- added by bonzini (Updated the item)
  • -unavailable- added by bens (Posted a comment)

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.


    Please enter the title of George Orwell's famous dystopian book (it's a date):



    Follow 4 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Sat Feb 6 13:13:45 2010bonziniSummary\\w and [[:alnum:]] not equivalent in multibyte locale=>\w and [[:alnum:]] not equivalent in multibyte locale
    Fri Apr 20 12:16:58 2007bensCarbon-Copy-=>Added -unavailable-
    Fri Apr 20 12:15:41 2007bensCarbon-CopyRemoved -unavailable-=>-
    Fri Apr 20 12:14:25 2007bensCarbon-Copy-=>Added -unavailable-

    Back to the top

    Powered by Savane 3.1-cleanup1