buggrep - Bugs: bug #37600, grep -w cuts words on non-ascii

 
 

You are not allowed to post comments on this tracker with your current authentication level.

bug #37600: grep -w cuts words on non-ascii

Submitter:  Flammie Pirinen <flammie>
Submitted:  Fri 19 Oct 2012 02:10:23 AM UTC
   
 
Category:  None Severity:  3 - Normal
Item Group:  None Status:  None
Privacy:  Public Assigned to:  None
Open/Closed:  Open

Mon 24 Feb 2014 02:57:54 PM UTC, comment #1: 

Note that it also affects \b, \<, \> and \w.

With -P, you can use:


grep -P '(*UCP)\bxxx\b'


But note that you can't use grep -wP in combination with (*UCP) as (*UCP) as to be first and -w wraps the regexps inside \b...\b.

Stephane Chazelas <stephanechazelas>
Fri 19 Oct 2012 02:10:23 AM UTC, original submission:  

It seems that grep -w does not support non-ascii characters, at least for locale fi-FI.utf8:

$ cat > test
xxx
xxxä
xxxx
$ grep -w xxx test
xxx
xxxä

System is Gentoo Linux, stable, x86 with GNU glibc-2.14.1-r3 and following setup:

$ grep -V
grep (GNU grep) 2.12
$ locale
LANG=fi_FI.UTF-8
LC_CTYPE="fi_FI.UTF-8"
LC_NUMERIC="fi_FI.UTF-8"
LC_TIME="fi_FI.UTF-8"
LC_COLLATE="fi_FI.UTF-8"
LC_MONETARY="fi_FI.UTF-8"
LC_MESSAGES="fi_FI.UTF-8"
LC_PAPER="fi_FI.UTF-8"
LC_NAME="fi_FI.UTF-8"
LC_ADDRESS="fi_FI.UTF-8"
LC_TELEPHONE="fi_FI.UTF-8"
LC_MEASUREMENT="fi_FI.UTF-8"
LC_IDENTIFICATION="fi_FI.UTF-8"
LC_ALL=fi_FI.UTF-8


If this behaviour is intentional, the description of -w switch in documentation should be clarified. Since grep can well match ä to [:alpha:] class on my locale I would expect from following that ä is a "word constituent character":


       -w, --word-regexp
              Select  only  those  lines  containing  matches  that form whole
              words.  The test is that the matching substring must  either  be
              at  the  beginning  of  the  line,  or  preceded  by  a non-word
              constituent character.  Similarly, it must be either at the  end
              of  the  line  or  followed by a non-word constituent character.
              Word-constituent  characters  are  letters,  digits,   and   the
              underscore.

Flammie Pirinen <flammie>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by stephanechazelas (Posted a comment)
  • -email is unavailable- added by flammie (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

     

    No changes have been made to this item

    Back to the top

    Powered by Savane 3.13-f8d8.
    Corresponding source code