buggrep - Bugs: bug #33271, non-US characters and -o

 
 

You are not allowed to post comments on this tracker with your current authentication level.

bug #33271: non-US characters and -o

Submitted by:  Goli Mar <golimar>
Submitted on:  Tue 10 May 2011 11:02:17 AM UTC  
 
Category: NoneSeverity: 3 - Normal
Item Group: NoneStatus: Invalid
Privacy: PublicAssigned to: None
Open/Closed: Closed

Thu 19 May 2011 12:30:10 PM UTC, comment #3:

Thanks for confirming!

Paolo Bonzini <bonzini>
Thu 19 May 2011 09:09:21 AM UTC, comment #2:

I had:

LANG=en_US.UTF-8

It worked changing it to:

LANG=es_ES.latin9

It seems that my file is Latin-9: this is the hexdump for the conflicting line (Ñ seems to be ASCII 165 (octal 245), using just one byte)

              1. 071 060 062 064 060 056 060 060 073 061 060 062 061 137 120 117
              2. 9 0 2 4 0 . 0 0 ; 1 0 2 1 _ P O

0000010 132 125 105 114 117 137 245 103 060 137 060 061 067 071 073 101
0000010 Z U E L O 245 C 0 0 1 7 9 ; A
0000020 104 115 073 061 066 066 062 015 012
0000020 D M ; 1 6 6 2 \r \n
0000029

(checked with grep 2.6 and 2.8)

Goli Mar <golimar>
Wed 18 May 2011 05:49:09 PM UTC, comment #1:

What locale is your file in? And what is your locale ("env | grep ^L" gives the answer)?

For example, "." matches a full UTF-8 character in grep if your locale is UTF-8, and perhaps your "\~n" is not in UTF-8?

Paolo Bonzini <bonzini>
Tue 10 May 2011 11:02:17 AM UTC, original submission:

I use egrep and a pattern that contains '.', because I want to
match any character in that position. It actually finds a match, but when I try to extract only the matching part, I use -o, but it prints nothing:

$ cat file.csv | egrep '1021_POZUELO_.C0_0179'
90240.00;1021_POZUELO_ÑC0_0179;ADM;1662
$ cat file.csv | egrep -o '1021_POZUELO_.C0_0179'
$

This other example works fine:

$ cat file.csv | egrep '1021.POZUEL'
90240.00;1021_POZUELO_ÑC0_0179;ADM;1662
90242.00;1021_POZUELO_CM0_0181;ADM;1662
$ cat file.csv | egrep -o '1021.POZUEL'
1021_POZUEL
1021_POZUEL

It seems that the non-US character (Ñ) is causing a problem to the -o option. This happens with grep 2.6.3 in RHEL 6 x86_64.
I tried with grep 2.7 and it's worse: it never matches the non-US character, with or without -o

$ cat file.csv | grep-2.7/src/egrep '1021_POZUELO_.'
90242.00;1021_POZUELO_CM0_0181;ADM;1662
$ cat file.csv | grep-2.7/src/egrep '1021_POZUELO_'
90240.00;1021_POZUELO_ÑC0_0179;ADM;1662
90242.00;1021_POZUELO_CM0_0181;ADM;1662

Goli Mar <golimar>

 

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by bonzini (Posted a comment)
  • -unavailable- added by golimar (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 3 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Thu 19 May 2011 12:30:10 PM UTCbonziniStatusNeed Info=>Invalid
      Open/ClosedOpen=>Closed
    Wed 18 May 2011 05:49:09 PM UTCbonziniStatusNone=>Need Info

    Back to the top


    Powered by Savane 3.1-cleanup1