bugfindutils - Bugs: bug #26701, if there are any funny characters,...

 
 

bug #26701: if there are any funny characters, * no longer matches in certain locales

Submitter:  jidanni
Submitted:  Wed 27 May 2009 09:20:32 PM UTC
   
 
Category:  find Severity:  3 - Normal
Item Group:  None Status:  Confirmed
Privacy:  Public Assigned to:  ericb
Originator Name:  Open/Closed:  Open
Release:  None Fixed Release:  4.4.1
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Mon 02 Sep 2019 10:14:12 PM UTC, comment #11: 

https://pubs.opengroup.org/onlinepubs/9699919799/functions/fnmatch.html talks about matching characters.   I'm not clear on whether "*" is required to match a byte which is part of a sequence which does not encode a valid character in the current encoding, or for that matter whether it is required not to.

ISTR that gnulib works the way it does because the relevant byte sequences don't encode a "character" in the selected encoding.  

Eric, (I'm assigning the bug to you to check this) is this consistent with your understanding?


James Youngman <jay>
Group administrator
Sat 08 Sep 2012 09:25:58 AM UTC, comment #10: 

Please ignore my previous comment, I forgot to call setlocale. So my system fnmatch fails here, too.

Lars Noschinski <cebewee>
Sat 08 Sep 2012 08:17:59 AM UTC, comment #9: 

Also, it seems that on my system, find does not use gnulib's fnmatch (as there is no fnmatch.o file lying around after build), and the system fnmatch works fine, i.e. the test program outputs "Match!" in all the locales I tested (en_US.UTF-8, de_DE.UTF-8, C).


$ cat /home/lars/tmp/fnmatch/src/testfnm.c
#include <config.h>
#include <stdio.h>
#include "fnmatch.h"

int main(void) {
    char *str = "Die_Toten_Hosen_-_Mein_groe\337ter_Feind.ogg";
    char pat = "*Feind";
    int r = fnmatch(pat, str, 0);

    printf("STRING: %s\n", str);
    printf("PATTERN: %s\n", pat);
    if (r == 0)
        printf("Match!\n");
    else if (r == FNM_NOMATCH)
        printf("No match!\n");
    else
        printf("Error %d", r);
    return 0;
}

Lars Noschinski <cebewee>
Sat 08 Sep 2012 06:33:27 AM UTC, comment #8: 

This bug is still there in findutils-4.5.10. While looking for the current state of fnmatch in gnulib, I found that you encountered this problem already in 2005 and from the state of discussion, it seemed that it was fixed back then?

http://lists.gnu.org/archive/html/bug-gnulib/2005-06/msg00029.html

(I have no idea about automake and hence was not being able to test gnulib's current fnmatch implementation).

Lars Noschinski <cebewee>
Sat 10 Apr 2010 11:24:56 PM UTC, comment #7: 

Yes, this looks like a limitation in the gnulib (current as of just now) gnmatch implementation.  Still occurs in find-4.5.8.

/tmp/testcase$ LC_ALL=zh_TW.UTF-8 /home/james/source/GNU/findutils/git/gnu/compile/64/find/find oo -name '*'|nl
     1 oo
     2 oo/Thumbs.db
/tmp/testcase$ LC_ALL=zh_TW.UTF-8 /home/james/source/GNU/findutils/git/gnu/compile/64/find/find oo |nl
     1 oo
     2 oo/��+R+�+ЪԦm�W�d++.pdf
     3 oo/��+RRGB.jpg
     4 oo/��+R����-���F�|+��ʽO+@�p.ai
     5 oo/�]�p�z�+.pdf
     6 oo/Thumbs.db
     7 oo/��+R+�+�_A2.pdf
     8 oo/��+R+�+��U��++.pdf

(reminder: if you don't have that charset installed, you will not be able to reproduce the problem)

James Youngman <jay>
Group administrator
Thu 01 Apr 2010 05:09:25 AM UTC, comment #6: 

All I know is the bug is on Debian extreme latest.
find (GNU findutils) 4.4.2

If it does exist for you in 4.4.2 but not later, then it is fixed.

However I just use Debian so don't send source.

jidanni
Thu 01 Apr 2010 05:05:50 AM UTC, comment #5: 

Don't you see the attached test case files?

Here's even a new test case not in a file:
set -eu
n=/tmp/oo
mkdir -p $n; cd $n
#recode qp <<! SORRY MAN can't figure out recode syntax
perl -MMIME::QuotedPrint -nwle 'print decode_qp($_);' <<!|xargs touch
Thumbs.db
=A6]=A1p=A6z=AE+.pdf
=A6=FE+R+=F0+=D0_A2.pdf
=A6=FE+R+=F0+=D0=AA=D4=A6m=A6W=A2d++.pdf
=A6=FE+R+=F0+=D0=F1U=A9=B3++.pdf
=A6=FE+RRGB.jpg
=A6=FE+R=A1=FD=A6=A4-=AA=B5=BCF=A6|+=B6=A6=CA=BDO+@=A9p.ai
!
set -x
ls|nl
LC_ALL=zh_TW.UTF-8 find -name \*|nl
LC_ALL=C find -name \*|nl
find --version|head -n 1

jidanni
Wed 31 Mar 2010 07:40:28 AM UTC, comment #4: 

Please provide a short shell script which reproduces the problem.

James Youngman <jay>
Group administrator
Fri 04 Dec 2009 05:28:07 PM UTC, comment #3: 

All that's lacking here is a short shell script to reproduce the problem on a recent findutils version (for example 4.5.5).

James Youngman <jay>
Group administrator
Mon 16 Nov 2009 07:19:16 PM UTC, comment #2: 

I too have come across this problem when writing scripts to convert my m4a files into mp3's. The files were generated on windows but behave the same regardless of fstype.

find (GNU findutils) 4.4.2 on ubuntu 9.10
LANG=en_US.UTF-8

The "?" map to various special German characters (umlauts, etc.).  I have attached a tar file.

Examples:

find .
.
./1-01 Prelude.m4a
./1-08 Das sch�ne Fest, Johannistag.m4a
./2-04 La� sehn, ob Meister Sachs zu H.m4a
./2-09 �ble Dinge, die ich da merk.m4a
./3-06 Gr�� Gott, mein Junker!.m4a
./4-01 Das Gedicht_ hier lie� ich's.m4a
./4-02 Gr�� Gott, mein Evchen!.m4a
./4-03 Weilten die Sterne im liebliche.m4a

find . -name "*m4a"
./1-01 Prelude.m4a
./4-03 Weilten die Sterne im liebliche.m4a



(file #19070)

John Bonnen <jbonnen>
Sun 12 Jul 2009 10:10:29 AM UTC, comment #1: 

This might be a limitation of the fnmatch() implementation, but even if so it may well be possible to do better here.  The first thing we'll need is a reproducible test case (I can probably convert that into a regression test case).

Could you provide a short shell script which generates a file with a suitable name and demonstrates the problem?

James Youngman <jay>
Group administrator
Wed 27 May 2009 09:20:32 PM UTC, original submission:  

If there are any funny characters, * no longer matches in certain locales:

$ LC_ALL=zh_TW.UTF-8 find -name \*|wc -l
2
$ LC_ALL=C find -name \*|wc -l
8
$ find --version
find (GNU findutils) 4.4.1
$ ls|qp-encode
Thumbs.db
=A6]=A1p=A6z=AE+.pdf
=A6=FE+R+=F0+=D0_A2.pdf
=A6=FE+R+=F0+=D0=AA=D4=A6m=A6W=A2d++.pdf
=A6=FE+R+=F0+=D0=F1U=A9=B3++.pdf
=A6=FE+RRGB.jpg
=A6=FE+R=A1=FD=A6=A4-=AA=B5=BCF=A6|+=B6=A6=CA=BDO+@=A9p.ai

jidanni

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #19070:  testcase.tar added by jbonnen (10KiB - application/x-tar)
file #18418:  test.sh added by jidanni (393B - application/x-sh - Run This Test Case)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by grawity
  • -email is unavailable- added by cebewee (Posted a comment)
  • -email is unavailable- added by jbonnen (Updated the item)
  • -email is unavailable- added by jay (Posted a comment)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 8 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2019-09-02 jay Assigned tojay ericb
    2015-01-24 grawity Carbon-Copy- Added grawity
    2010-10-07 jay StatusNone Confirmed
    2010-10-07 jay StatusNeed Info None
    2009-11-16 jbonnen Attached File- Added testcase.tar, #19070
    2009-07-12 jidanni Attached File- Added test.sh, #18418
    2009-07-12 jay StatusNone Need Info
        Assigned toNone jay

    Back to the top

    Powered by Savane 3.13-758e.
    Corresponding source code