buggrep - Bugs: bug #37600, grep -w cuts words on non-ascii

 
 

You are not allowed to post comments on this tracker with your current authentification level.

bug #37600: grep -w cuts words on non-ascii

Submitted by:  Flammie Pirinen <flammie>
Submitted on:  Fri 19 Oct 2012 02:10:23 AM UTC  
 
Category: NoneSeverity: 3 - Normal
Item Group: NoneStatus: None
Privacy: PublicAssigned to: None
Open/Closed: Open

Mon 24 Feb 2014 02:57:54 PM UTC, comment #1:

Note that it also affects \b, \<, \> and \w.

With -P, you can use:

But note that you can't use grep -wP in combination with (UCP) as (UCP) as to be first and -w wraps the regexps inside \b...\b.

Stephane Chazelas <stephanechazelas>
Fri 19 Oct 2012 02:10:23 AM UTC, original submission:

It seems that grep -w does not support non-ascii characters, at least for locale fi-FI.utf8:

$ cat > test
xxx
xxxä
xxxx
$ grep -w xxx test
xxx
xxxä

System is Gentoo Linux, stable, x86 with GNU glibc-2.14.1-r3 and following setup:

$ grep -V
grep (GNU grep) 2.12
$ locale
LANG=fi_FI.UTF-8
LC_CTYPE="fi_FI.UTF-8"
LC_NUMERIC="fi_FI.UTF-8"
LC_TIME="fi_FI.UTF-8"
LC_COLLATE="fi_FI.UTF-8"
LC_MONETARY="fi_FI.UTF-8"
LC_MESSAGES="fi_FI.UTF-8"
LC_PAPER="fi_FI.UTF-8"
LC_NAME="fi_FI.UTF-8"
LC_ADDRESS="fi_FI.UTF-8"
LC_TELEPHONE="fi_FI.UTF-8"
LC_MEASUREMENT="fi_FI.UTF-8"
LC_IDENTIFICATION="fi_FI.UTF-8"
LC_ALL=fi_FI.UTF-8

If this behaviour is intentional, the description of -w switch in documentation should be clarified. Since grep can well match ä to [:alpha:] class on my locale I would expect from following that ä is a "word constituent character":

-w, --word-regexp
Select only those lines containing matches that form whole
words. The test is that the matching substring must either be
at the beginning of the line, or preceded by a non-word
constituent character. Similarly, it must be either at the end
of the line or followed by a non-word constituent character.
Word-constituent characters are letters, digits, and the
underscore.

Flammie Pirinen <flammie>

 

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by stephanechazelas (Posted a comment)
  • -unavailable- added by flammie (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    No Changes Have Been Made to This Item

    Back to the top


    Powered by Savane 3.1-cleanup