buggrep - Bugs: bug #37600, grep -w cuts words on non-ascii

 
 

You are not allowed to post comments on this tracker with your current authentication level.

bug #37600: grep -w cuts words on non-ascii

Submitter:  Flammie Pirinen <flammie>
Submitted:  Fri 19 Oct 2012 02:10:23 AM UTC
   
 
Category:  None Severity:  3 - Normal
Priority:  5 - Normal Item Group:  None
Status:  None Privacy:  Public
Assigned to:  None Open/Closed:  Open

Discussion

Mon 24 Feb 2014 02:57:54 PM UTC, comment #1: 

Note that it also affects \b, \<, \> and \w.

With -P, you can use:


grep -P '(*UCP)\bxxx\b'


But note that you can't use grep -wP in combination with (*UCP) as (*UCP) as to be first and -w wraps the regexps inside \b...\b.

Stephane Chazelas <stephanechazelas>
Fri 19 Oct 2012 02:10:23 AM UTC, original submission:  

It seems that grep -w does not support non-ascii characters, at least for locale fi-FI.utf8:

$ cat > test
xxx
xxxä
xxxx
$ grep -w xxx test
xxx
xxxä

System is Gentoo Linux, stable, x86 with GNU glibc-2.14.1-r3 and following setup:

$ grep -V
grep (GNU grep) 2.12
$ locale
LANG=fi_FI.UTF-8
LC_CTYPE="fi_FI.UTF-8"
LC_NUMERIC="fi_FI.UTF-8"
LC_TIME="fi_FI.UTF-8"
LC_COLLATE="fi_FI.UTF-8"
LC_MONETARY="fi_FI.UTF-8"
LC_MESSAGES="fi_FI.UTF-8"
LC_PAPER="fi_FI.UTF-8"
LC_NAME="fi_FI.UTF-8"
LC_ADDRESS="fi_FI.UTF-8"
LC_TELEPHONE="fi_FI.UTF-8"
LC_MEASUREMENT="fi_FI.UTF-8"
LC_IDENTIFICATION="fi_FI.UTF-8"
LC_ALL=fi_FI.UTF-8


If this behaviour is intentional, the description of -w switch in documentation should be clarified. Since grep can well match ä to [:alpha:] class on my locale I would expect from following that ä is a "word constituent character":


       -w, --word-regexp
              Select  only  those  lines  containing  matches  that form whole
              words.  The test is that the matching substring must  either  be
              at  the  beginning  of  the  line,  or  preceded  by  a non-word
              constituent character.  Similarly, it must be either at the  end
              of  the  line  or  followed by a non-word constituent character.
              Word-constituent  characters  are  letters,  digits,   and   the
              underscore.

Flammie Pirinen <flammie>

 

Attached Files

This item currently has no attached files.

 

Dependencies

This item does not depend on any other items.

No items depend on this one.

 

Mail Notification Carbon-Copy List

Carbon-Copy List
  • -email is unavailable- added by stephanechazelas (Posted a comment)
  • -email is unavailable- added by flammie (Submitted the item)
  •  

    Votes

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

     

    History

    No changes have been made to this item

    Back to the top

    Powered by Savane 3.16-2753.
    Corresponding source code