bugfindutils - Bugs: bug #8455, anomaly on large xfs filesystems

 
 

bug #8455: anomaly on large xfs filesystems

Submitter:  Andreas Metzler <ametzler>
Submitted:  Mon 05 Apr 2004 03:29:16 PM UTC
   
 
Category:  find Severity:  3 - Normal
Item Group:  None Status:  Invalid
Privacy:  Public Assigned to:  jay
Originator Name:  Open/Closed:  Closed
Release:  None Fixed Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Sun 05 Dec 2004 02:46:05 PM UTC, comment #9: 

Hence I don't think this is a bug.

James Youngman <jay>
Group administrator
Sun 05 Dec 2004 02:41:04 PM UTC, comment #8: 

Hmm.  See also http://seclists.org/lists/linux-kernel/2000/May/5068.html

The IRIX documentation for exports(4) says:

32bitclients

Causes the server to mask off the high order 32 bits of directory cookies in NFS version 3 directory operations.  This option may be required when clients run 32-bit  operating systems that assume the entire cookie is contained in 32 bits and reject responses containing version 3 cookies with high bits on.  IRIX 5.3 and Solaris 2.5 are examples of 32-bit operating systems with this behavior, which produces error messages like "Cannot decode response" on directory operations.  XFS filesystems on the server can generate cookies with high bits on.  Exporting filesystems with the 32bitclients option causes these bits to be masked and prevents error messages.

James Youngman <jay>
Group administrator
Sun 05 Dec 2004 02:18:17 PM UTC, comment #7: 

I have just read http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=180065 which seems to state that the problem is in fact a Linux kernel bug  for which a fix exists, and there is even a glibc workaround patch for it, too.


James Youngman <jay>
Group administrator
Thu 02 Dec 2004 10:02:00 AM UTC, comment #6: 
Tim Waugh <twaugh>
Thu 02 Dec 2004 09:11:54 AM UTC, comment #5: 
James Youngman <jay>
Group administrator
Mon 22 Nov 2004 12:45:21 AM UTC, comment #4: 

I've created a local XFS filesystem with a couple of million files in it (one directory has over 900,000 entries).  find searches it in about 15 seconds without failing.  I'm not using NFS though.

James Youngman <jay>
Group administrator
Sat 02 Oct 2004 10:48:01 AM UTC, comment #3: 

OK, having looked at the trace output, I still suspect that it is savedir() which is failing.  I have a small number of questions.
1. If you use "tar" to copy the directory tree to a local filesystem, can you reproduce the problem?  Don't forgert to use the "p" option when unpacking.

The other checks involve getting hold of a copy of the source.  If you're not confident with compiling source code, don't worry about it.  Otherwise, I have a couple of other suggested tests:

a) Change error() [in file error.c] so that it calls abort().  Then use GDB to get a stack backtrace from the coredump.   Send me a copy of the stack backtrace.  For the lower levels of the backtrace (say 3 as a guideline) show me which line that corresponds to (the will avoid problems caused by us looking at different versions of the code). 

b) Use GDB to single-step through savedir() to figure out why it fails - if indeed (b) shows that it has failed.

Sorry to make you do all the work, but I can't reproduce the problem here.

James Youngman <jay>
Group administrator
Mon 30 Aug 2004 03:54:00 PM UTC, comment #2: 

On Mon, Aug 09, 2004 at 01:01:12PM +0200, Andreas Metzler wrote:

> On 2004-04-01 Andrew Reid <Andrew.Reid@nist.gov> wrote:
> > Package: findutils
> > Version: 4.1.20-3
>
> > I am having a rather strange difficulty with the "find" command.
>
> > For many paths on my system, including all paths in my
> > account, I get:
>
> > > # find <path> -print
> > > find: <path>: Value too large for defined data type
> [...]
>


  Thanks for the reply and sorry about my delay in replying -- partially
it's vacation season, and partially I've just been doing other stuff.

> While browsing upstream's bugtracker where I had forwarded the bug to
> I found this in
> http://savannah.gnu.org/bugs/index.php?func=detailitem&item_id=8455
>
> -----------------------
> The likely difference between the two versions of find is that they
> use a different version of gnulib, and the updated version is now
> sensitive to this problem. Alternatively, maybe it's a glibc
> difference. Certainly the differences between the find application
> code is minimal.
>
> Looking at the ltrace output, it looks like readdir() is failing. The
> likely bit of affected code is gnulib's savedir.c. That certainly
> should faiil if readdir() fails.
>
> I suspect that somewhere in there is a filename whose base name is
> more than 256 characters. To find out, please try this (on either the
> client or the server):
>
> ls -R1 | awk '(length > 200) {print length, $0;}' | sort -n | tail


  This suspicion is not confirmed, alas -- the awk returns nothing
with length > 200.  To check that it worked at all, I ratcheted
down the number, and I start to get lots of stuff once it's
less than around 50.  Nothing longer than 100.  I made no attempt
to refine it beyond that.

> Try picking the last filename from the above and invoking find
> directly on it. The idea here is to pin things down to one path name
> that successfully reproduces the error.


  I appreciate the need for diagnosing under controlled conditions,
so I've picked a directory to run it, which I will use for future
tests, if any...

> Lastly, the output of "ltrace" isn't very helpful about the data
> that's returned from the system calls. What do you get if you use
> "strace"? What about "ltrace -S"?


  Here's another crack at it -- in case you're curious, this was
test code for what, exactly, Python does to its internal search
paths at start-up time.
  Also, recall that the filesystem being examined is an SGI XFS
filesystem which is NFS mounted by the Debian Sarge linux box
which is actually running the "find" command.

  The directory:

> poppins[reida]: ls -al
> total 24
> drwxr-xr-x   2 reida user   95 Feb  3  2004 ./
> drwxr-xr-x  22 reida user 4096 Aug 25 20:35 ../
> -rw-r--r--   1 reida user   20 Feb  3  2004 stuff.py
> -rw-r--r--   1 reida user  229 Feb  3  2004 stuff.pyc
> -rw-r--r--   1 reida user   44 Feb  3  2004 stuff.py~
> -rwxr-xr--   1 reida user  230 Feb  3  2004 test.py*
> -rw-r--r--   1 reida user   95 Feb  3  2004 test.py~

  Debian Woody, with strace output afterwards.

> jeeves[reida]: find . -name "*.py"
> ./test.py
> ./stuff.py


> jeeves[reida]: strace find . -name "*.py"
> execve("/usr/bin/find", ["find", ".", "-name", "*.py"], [/* 35 vars */]) = 0
> uname({sys="Linux", node="jeeves", ...}) = 0
> brk(0)                                  = 0x8053d54
> open("/etc/ld.so.preload", O_RDONLY)    = -1 ENOENT (No such file or directory)
> open("/usr/site/lib/i686/mmx/libc.so.6", O_RDONLY) = -1 ENOENT (No such file or directory)
> stat64("/usr/site/lib/i686/mmx", 0xbfffef04) = -1 ENOENT (No such file or directory)
> open("/usr/site/lib/i686/libc.so.6", O_RDONLY) = -1 ENOENT (No such file or directory)
> stat64("/usr/site/lib/i686", 0xbfffef04) = -1 ENOENT (No such file or directory)
> open("/usr/site/lib/mmx/libc.so.6", O_RDONLY) = -1 ENOENT (No such file or directory)
> stat64("/usr/site/lib/mmx", 0xbfffef04) = -1 ENOENT (No such file or directory)
> open("/usr/site/lib/libc.so.6", O_RDONLY) = -1 ENOENT (No such file or directory)
> stat64("/usr/site/lib", {st_mode=S_IFDIR|0755, st_size=776, ...}) = 0
> open("/etc/ld.so.cache", O_RDONLY)      = 3
> fstat64(3, {st_mode=S_IFREG|0644, st_size=41861, ...}) = 0
> old_mmap(NULL, 41861, PROT_READ, MAP_PRIVATE, 3, 0) = 0x40014000
> close(3)                                = 0
> open("/lib/libc.so.6", O_RDONLY)        = 3
> read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\30\222"..., 1024) = 1024
> fstat64(3, {st_mode=S_IFREG|0755, st_size=1153784, ...}) = 0
> old_mmap(NULL, 1166560, PROT_READ|PROT_EXEC, MAP_PRIVATE, 3, 0) = 0x4001f000
> mprotect(0x40132000, 40160, PROT_NONE)  = 0
> old_mmap(0x40132000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 3, 0x113000) = 0x40132000
> old_mmap(0x40138000, 15584, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x40138000
> close(3)                                = 0
> munmap(0x40014000, 41861)               = 0
> brk(0)                                  = 0x8053d54
> brk(0x8053d7c)                          = 0x8053d7c
> brk(0x8054000)                          = 0x8054000
> time(NULL)                              = 1093877075
> open(".", O_RDONLY|O_LARGEFILE)         = 3
> fchdir(3)                               = 0
> lstat64(".", {st_mode=S_IFDIR|0755, st_size=95, ...}) = 0
> lstat64(".", {st_mode=S_IFDIR|0755, st_size=95, ...}) = 0
> chdir(".")                              = 0
> lstat64(".", {st_mode=S_IFDIR|0755, st_size=95, ...}) = 0
> lstat64(".", {st_mode=S_IFDIR|0755, st_size=95, ...}) = 0
> brk(0x8055000)                          = 0x8055000
> open("/dev/null", O_RDONLY|O_NONBLOCK|O_DIRECTORY) = -1 ENOTDIR (Not a directory)
> open(".", O_RDONLY|O_NONBLOCK|O_LARGEFILE|O_DIRECTORY) = 4
> fstat64(4, {st_mode=S_IFDIR|0755, st_size=95, ...}) = 0
> fcntl64(4, F_SETFD, FD_CLOEXEC)         = 0
> brk(0x8058000)                          = 0x8058000
> getdents64(0x4, 0x8054148, 0x2000, 0)   = 208
> getdents64(0x4, 0x8054148, 0x2000, 0)   = 0
> close(4)                                = 0
> lstat64(".", {st_mode=S_IFDIR|0755, st_size=95, ...}) = 0
> fstat64(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 0), ...}) = 0
> old_mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x40014000
> write(1, "./test.py\n", 10./test.py)             = 10
> write(1, "./stuff.py\n", 11./stuff.py)            = 11
> fchdir(3)                               = 0
> munmap(0x40014000, 4096)                = 0
> _exit(0)                                = ?



    ... then Debian Sarge:  Since this is the system with the problem,
I have included both strace and ltrace -S output -- they seem to not
quite be the same, so you may of course use whichever is most useful
to you.


> poppins[reida]: find . -name "*.py"
> find: .: Value too large for defined data type




> poppins[reida]: strace find . -name "*.py"
> execve("/usr/bin/find", ["find", ".", "-name", "*.py"], [/* 46 vars */]) = 0
> uname({sys="Linux", node="poppins", ...}) = 0
> brk(0)                                  = 0x80545b4
> old_mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x40017000
> access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
> open("/etc/ld.so.preload", O_RDONLY)    = -1 ENOENT (No such file or directory)
> open("/usr/site/lib/tls/i686/mmx/cmov/libc.so.6", O_RDONLY) = -1 ENOENT (No such file or directory)
> stat64("/usr/site/lib/tls/i686/mmx/cmov", 0xbffff0a8) = -1 ENOENT (No such file or directory)
> open("/usr/site/lib/tls/i686/mmx/libc.so.6", O_RDONLY) = -1 ENOENT (No such file or directory)
> stat64("/usr/site/lib/tls/i686/mmx", 0xbffff0a8) = -1 ENOENT (No such file or directory)
> open("/usr/site/lib/tls/i686/cmov/libc.so.6", O_RDONLY) = -1 ENOENT (No such file or directory)
> stat64("/usr/site/lib/tls/i686/cmov", 0xbffff0a8) = -1 ENOENT (No such file or directory)
> open("/usr/site/lib/tls/i686/libc.so.6", O_RDONLY) = -1 ENOENT (No such file or directory)
> stat64("/usr/site/lib/tls/i686", 0xbffff0a8) = -1 ENOENT (No such file or directory)
> open("/usr/site/lib/tls/mmx/cmov/libc.so.6", O_RDONLY) = -1 ENOENT (No such file or directory)
> stat64("/usr/site/lib/tls/mmx/cmov", 0xbffff0a8) = -1 ENOENT (No such file or directory)
> open("/usr/site/lib/tls/mmx/libc.so.6", O_RDONLY) = -1 ENOENT (No such file or directory)
> stat64("/usr/site/lib/tls/mmx", 0xbffff0a8) = -1 ENOENT (No such file or directory)
> open("/usr/site/lib/tls/cmov/libc.so.6", O_RDONLY) = -1 ENOENT (No such file or directory)
> stat64("/usr/site/lib/tls/cmov", 0xbffff0a8) = -1 ENOENT (No such file or directory)
> open("/usr/site/lib/tls/libc.so.6", O_RDONLY) = -1 ENOENT (No such file or directory)
> stat64("/usr/site/lib/tls", 0xbffff0a8) = -1 ENOENT (No such file or directory)
> open("/usr/site/lib/i686/mmx/cmov/libc.so.6", O_RDONLY) = -1 ENOENT (No such file or directory)
> stat64("/usr/site/lib/i686/mmx/cmov", 0xbffff0a8) = -1 ENOENT (No such file or directory)
> open("/usr/site/lib/i686/mmx/libc.so.6", O_RDONLY) = -1 ENOENT (No such file or directory)
> stat64("/usr/site/lib/i686/mmx", 0xbffff0a8) = -1 ENOENT (No such file or directory)
> open("/usr/site/lib/i686/cmov/libc.so.6", O_RDONLY) = -1 ENOENT (No such file or directory)
> stat64("/usr/site/lib/i686/cmov", 0xbffff0a8) = -1 ENOENT (No such file or directory)
> open("/usr/site/lib/i686/libc.so.6", O_RDONLY) = -1 ENOENT (No such file or directory)
> stat64("/usr/site/lib/i686", 0xbffff0a8) = -1 ENOENT (No such file or directory)
> open("/usr/site/lib/mmx/cmov/libc.so.6", O_RDONLY) = -1 ENOENT (No such file or directory)
> stat64("/usr/site/lib/mmx/cmov", 0xbffff0a8) = -1 ENOENT (No such file or directory)
> open("/usr/site/lib/mmx/libc.so.6", O_RDONLY) = -1 ENOENT (No such file or directory)
> stat64("/usr/site/lib/mmx", 0xbffff0a8) = -1 ENOENT (No such file or directory)
> open("/usr/site/lib/cmov/libc.so.6", O_RDONLY) = -1 ENOENT (No such file or directory)
> stat64("/usr/site/lib/cmov", 0xbffff0a8) = -1 ENOENT (No such file or directory)
> open("/usr/site/lib/libc.so.6", O_RDONLY) = -1 ENOENT (No such file or directory)
> stat64("/usr/site/lib", {st_mode=S_IFDIR|0755, st_size=776, ...}) = 0
> open("/etc/ld.so.cache", O_RDONLY)      = 3
> fstat64(3, {st_mode=S_IFREG|0644, st_size=65341, ...}) = 0
> old_mmap(NULL, 65341, PROT_READ, MAP_PRIVATE, 3, 0) = 0x40018000
> close(3)                                = 0
> access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
> open("/lib/libc.so.6", O_RDONLY)        = 3
> read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\200^\1"..., 512) = 512
> fstat64(3, {st_mode=S_IFREG|0644, st_size=1243856, ...}) = 0
> old_mmap(NULL, 1254020, PROT_READ|PROT_EXEC, MAP_PRIVATE, 3, 0) = 0x40028000
> old_mmap(0x40150000, 32768, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 3, 0x127000) = 0x40150000
> old_mmap(0x40158000, 8836, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x40158000
> close(3)                                = 0
> munmap(0x40018000, 65341)               = 0
> brk(0)                                  = 0x80545b4
> brk(0x80755b4)                          = 0x80755b4
> brk(0)                                  = 0x80755b4
> brk(0x8076000)                          = 0x8076000
> time(NULL)                              = 1093877299
> open(".", O_RDONLY|O_LARGEFILE)         = 3
> fchdir(3)                               = 0
> lstat64(".", {st_mode=S_IFDIR|0755, st_size=20480, ...}) = 0
> lstat64(".", {st_mode=S_IFDIR|0755, st_size=20480, ...}) = 0
> chdir(".")                              = 0
> lstat64(".", {st_mode=S_IFDIR|0755, st_size=20480, ...}) = 0
> lstat64(".", {st_mode=S_IFDIR|0755, st_size=20480, ...}) = 0
> open("/dev/null", O_RDONLY|O_NONBLOCK|O_DIRECTORY) = -1 ENOTDIR (Not a directory)
> open(".", O_RDONLY|O_NONBLOCK|O_LARGEFILE|O_DIRECTORY) = 4
> fstat64(4, {st_mode=S_IFDIR|0755, st_size=20480, ...}) = 0
> fcntl64(4, F_SETFD, FD_CLOEXEC)         = 0
> getdents64(4, /* 219 entries */, 8192)  = 8168
> _llseek(4, 2144606367, [2144606367], SEEK_SET) = 0
> getdents64(4, /* 144 entries */, 8192)  = 5584
> close(4)                                = 0
> brk(0)                                  = 0x8076000
> brk(0)                                  = 0x8076000
> brk(0x8075000)                          = 0x8075000
> brk(0)                                  = 0x8075000
> write(2, "find: ", 6find: )                   = 6
> write(2, ".", 1.)                        = 1
> write(2, ": Value too large for defined da"..., 39: Value too large for defined data type) = 39
> write(2, "\n", 1 )                       = 1
> fchdir(3)                               = 0
> exit_group(1)                           = ?




> poppins[reida]: ltrace -S find . -name "*.py"
> SYS_uname(0xbffff70c)                            = 0
> SYS_brk(NULL)                                    = 0x80545b4
> SYS_mmap(0xbffff4ec, 34, 0x40016c40, 4096, 565)  = 0x40017000
> SYS_access(0x40012de4, 0, 0x40016c40, 0x40016e78, 24) = -2
> SYS_open("/etc/ld.so.preload", 0, 010000226744)  = -2
> SYS_open("/usr/site/lib/tls/i686/mmx/cmov/"..., 0, 00) = -2
> SYS_stat64(0xbffff040, 0xbffff098, 0x40016c40, 0x40016e60, 2) = -2
> SYS_open("/usr/site/lib/tls/i686/mmx/libc."..., 0, 00) = -2
> SYS_stat64(0xbffff040, 0xbffff098, 0x40016c40, 0x40016e60, 2) = -2
> SYS_open("/usr/site/lib/tls/i686/cmov/libc"..., 0, 00) = -2
> SYS_stat64(0xbffff040, 0xbffff098, 0x40016c40, 0x40016e60, 2) = -2
> SYS_open("/usr/site/lib/tls/i686/libc.so.6", 0, 00) = -2
> SYS_stat64(0xbffff040, 0xbffff098, 0x40016c40, 0x40016e60, 2) = -2
> SYS_open("/usr/site/lib/tls/mmx/cmov/libc."..., 0, 00) = -2
> SYS_stat64(0xbffff040, 0xbffff098, 0x40016c40, 0x40016e60, 2) = -2
> SYS_open("/usr/site/lib/tls/mmx/libc.so.6", 0, 00) = -2
> SYS_stat64(0xbffff040, 0xbffff098, 0x40016c40, 0x40016e60, 2) = -2
> SYS_open("/usr/site/lib/tls/cmov/libc.so.6", 0, 00) = -2
> SYS_stat64(0xbffff040, 0xbffff098, 0x40016c40, 0x40016e60, 2) = -2
> SYS_open("/usr/site/lib/tls/libc.so.6", 0, 00)   = -2
> SYS_stat64(0xbffff040, 0xbffff098, 0x40016c40, 0x40016e60, 2) = -2
> SYS_open("/usr/site/lib/i686/mmx/cmov/libc"..., 0, 00) = -2
> SYS_stat64(0xbffff040, 0xbffff098, 0x40016c40, 0x40016e60, 2) = -2
> SYS_open("/usr/site/lib/i686/mmx/libc.so.6", 0, 00) = -2
> SYS_stat64(0xbffff040, 0xbffff098, 0x40016c40, 0x40016e60, 2) = -2
> SYS_open("/usr/site/lib/i686/cmov/libc.so."..., 0, 00) = -2
> SYS_stat64(0xbffff040, 0xbffff098, 0x40016c40, 0x40016e60, 2) = -2
> SYS_open("/usr/site/lib/i686/libc.so.6", 0, 00)  = -2
> SYS_stat64(0xbffff040, 0xbffff098, 0x40016c40, 0x40016e60, 2) = -2
> SYS_open("/usr/site/lib/mmx/cmov/libc.so.6", 0, 00) = -2
> SYS_stat64(0xbffff040, 0xbffff098, 0x40016c40, 0x40016e60, 2) = -2
> SYS_open("/usr/site/lib/mmx/libc.so.6", 0, 00)   = -2
> SYS_stat64(0xbffff040, 0xbffff098, 0x40016c40, 0x40016e60, 2) = -2
> SYS_open("/usr/site/lib/cmov/libc.so.6", 0, 00)  = -2
> SYS_stat64(0xbffff040, 0xbffff098, 0x40016c40, 0x40016e60, 2) = -2
> SYS_open("/usr/site/lib/libc.so.6", 0, 00)       = -2
> SYS_stat64(0xbffff040, 0xbffff098, 0x40016c40, 0x40016e60, 2) = 0
> SYS_open("/etc/ld.so.cache", 0, 00)              = 3
> SYS_fstat64(3, 0xbfffefe8, 0x40016c40, 0x40016e60, 2) = 0
> SYS_mmap(0xbfffefc8, 0, 0x40016c40, 3, 0x40016d30) = 0x40018000
> SYS_close(3)                                     = 0
> SYS_access(0x40012de4, 0, 0x40016c40, 0, 524288) = -2
> SYS_open("/lib/libc.so.6", 0, 027777775257)      = 3
> SYS_read(3, "\177ELF\001\001\001", 512)          = 512
> SYS_fstat64(3, 0xbffff098, 0x40016c40, 0x40016e60, 0) = 0
> SYS_mmap(0xbfffef1c, 3, 0x40016c40, 0x132284, 0xbfffef50) = 0x40028000
> SYS_mmap(0xbfffef1c, 0x40028000, 0x40016c40, 0x128000, 0xbfffef68) =
> 0x40150000
> SYS_mmap(0xbfffef1c, 8836, 0x40016c40, 50, 0xbfffef68) = 0x40158000
> SYS_close(3)                                     = 0
> SYS_munmap(0x40018000, 65341)                    = 0
> __libc_start_main(0x80496a0, 4, 0xbffff954, 0x8052610, 0x8052670
> <unfinished ...>
> setlocale(6, "")                                 = "C"
> bindtextdomain("findutils", "/usr/share/locale" <unfinished ...>
> SYS_brk(NULL)                                    = 0x80545b4
> SYS_brk(0x80755b4)                               = 0x80755b4
> SYS_brk(NULL)                                    = 0x80755b4
> SYS_brk(0x8076000)                               = 0x8076000
> <... bindtextdomain resumed> )                   = "/usr/share/locale"
> textdomain("findutils")                          = "findutils"
> time(NULL <unfinished ...>
> SYS_time(0, 0, 0x8052710, 0, 0xbffff954)         = 0x41333a1b
> <... time resumed> )                             = 1093876251
> getenv("FIND_BLOCK_SIZE")                        = NULL
> getenv("BLOCK_SIZE")                             = NULL
> getenv("POSIXLY_CORRECT")                        = NULL
> strchr("-!(),", '.')                             = NULL
> strchr("-!(),", '-')                             = "-!(),"
> malloc(56)                                       = 0x8054638
> strchr("-!(),", '-')                             = "-!(),"
> strcmp("!", "name")                              = -77
> strcmp("not", "name")                            = 14
> strcmp("(", "name")                              = -70
> strcmp(")", "name")                              = -69
> strcmp(",", "name")                              = -66
> strcmp("a", "name")                              = -13
> strcmp("amin", "name")                           = -13
> strcmp("and", "name")                            = -13
> strcmp("anewer", "name")                         = -13
> strcmp("atime", "name")                          = -13
> strcmp("cmin", "name")                           = -11
> strcmp("cnewer", "name")                         = -11
> strcmp("ctime", "name")                          = -11
> strcmp("daystart", "name")                       = -10
> strcmp("depth", "name")                          = -10
> strcmp("empty", "name")                          = -9
> strcmp("exec", "name")                           = -9
> strcmp("false", "name")                          = -8
> strcmp("fls", "name")                            = -8
> strcmp("follow", "name")                         = -8
> strcmp("fprint", "name")                         = -8
> strcmp("fprint0", "name")                        = -8
> strcmp("fprintf", "name")                        = -8
> strcmp("fstype", "name")                         = -8
> strcmp("gid", "name")                            = -7
> strcmp("group", "name")                          = -7
> strcmp("help", "name")                           = -6
> strcmp("-help", "name")                          = -65
> strcmp("ilname", "name")                         = -5
> strcmp("iname", "name")                          = -5
> strcmp("inum", "name")                           = -5
> strcmp("ipath", "name")                          = -5
> strcmp("iregex", "name")                         = -5
> strcmp("links", "name")                          = -2
> strcmp("lname", "name")                          = -2
> strcmp("ls", "name")                             = -2
> strcmp("maxdepth", "name")                       = -1
> strcmp("mindepth", "name")                       = -1
> strcmp("mmin", "name")                           = -1
> strcmp("mount", "name")                          = -1
> strcmp("mtime", "name")                          = -1
> strcmp("name", "name")                           = 0
> malloc(56)                                       = 0x8054678
> malloc(56)                                       = 0x80546b8
> malloc(56)                                       = 0x80546f8
> malloc(56)                                       = 0x8054738
> malloc(56)                                       = 0x8054778
> open64(".", 0, 01001242620 <unfinished ...>
> SYS_open(".", 32768, 00)                         = 3
> <... open64 resumed> )                           = 3
> fchdir(3 <unfinished ...>
> SYS_fchdir(3, 32768, 4, 0xbffffabb, 0xbffff954)  = 0
> <... fchdir resumed> )                           = 0
> __lxstat64(3, ".", 0x8054500 <unfinished ...>
> SYS_lstat64(0x8052865, 0x8054500, 0x40157550, 0x4015a008, 0x8054500) = 0
> <... __lxstat64 resumed> )                       = 0
> strchr("-!(),", '.')                             = NULL
> strlen(".")                                      = 1
> __lxstat64(3, ".", 0xbffff850 <unfinished ...>
> SYS_lstat64(0xbffffab9, 0xbffff850, 0x40157550, 0x4015a008, 0xbffff850)
> = 0
> <... __lxstat64 resumed> )                       = 0
> chdir("." <unfinished ...>
> SYS_chdir(0xbffffab9, 0xbffff850, 1, 0, 0xbffffab9) = 0
> <... chdir resumed> )                            = 0
> __lxstat64(3, ".", 0xbffff7f0 <unfinished ...>
> SYS_lstat64(0x8052865, 0xbffff7f0, 0x40157550, 0x4015a008, 0xbffff7f0) =
> 0
> <... __lxstat64 resumed> )                       = 0
> __lxstat64(3, ".", 0xbffff750 <unfinished ...>
> SYS_lstat64(0x8052865, 0xbffff750, 0x40157550, 0x4015a008, 0xbffff750) =
> 0
> <... __lxstat64 resumed> )                       = 0
> realloc(NULL, 512)                               = 0x80547b8
> fnmatch("*.py", ".", 4)                          = 1
> strlen(".")                                      = 1
> __errno_location()                               = 0x40157900
> opendir("." <unfinished ...>
> SYS_open("/dev/null", 67584, 010005272520)       = -20
> SYS_open(".", 100352, 010005272520)              = 4
> SYS_fstat64(4, 0xbffff54c, 0x40157550, 0x4015a008, 0xbffff54c) = 0
> SYS_fcntl64(4, 2, 1, 1, 0x40157550)              = 0
> <... opendir resumed> )                          = 0x80549c0
> malloc(512)                                      = 0x80569f8
> __errno_location()                               = 0x40157900
> readdir(0x80549c0 <unfinished ...>
> SYS_getdents64(4, 0x80549f0, 8192, 0x40157900, 0x40157550) = 208
> SYS__llseek(4, 0, 0x6c2c6bd7, 0xbffff51c, 0)     = 0
> <... readdir resumed> )                          = 0x80549f0
> readdir(0x80549c0)                               = 0x8054a00
> readdir(0x80549c0)                               = 0x8054a10
> strlen("stuff.py~")                              = 9
> memcpy(0x80569f8, "stuff.py~", 10)               = 0x80569f8
> readdir(0x80549c0)                               = 0x8054a28
> strlen("stuff.pyc")                              = 9
> memcpy(0x8056a02, "stuff.pyc", 10)               = 0x8056a02
> readdir(0x80549c0)                               = 0x8054a40
> strlen("test.py")                                = 7
> memcpy(0x8056a0c, "test.py", 8)                  = 0x8056a0c
> readdir(0x80549c0)                               = 0x8054a58
> strlen("test.py~")                               = 8
> memcpy(0x8056a14, "test.py~", 9)                 = 0x8056a14
> readdir(0x80549c0 <unfinished ...>
> SYS_getdents64(4, 0x80549f0, 8192, 0x40157900, 0x40157550) = 32
> <... readdir resumed> )                          = NULL
> __errno_location()                               = 0x40157900
> closedir(0x80549c0 <unfinished ...>
> SYS_close(4)                                     = 0
> <... closedir resumed> )                         = 0
> free(0x80569f8 <unfinished ...>
> SYS_brk(NULL)                                    = 0x8076000
> SYS_brk(NULL)                                    = 0x8076000
> SYS_brk(0x8075000)                               = 0x8075000
> SYS_brk(NULL)                                    = 0x8075000
> <... free resumed> )                             = <void>
> error(0, 75, 0x8053b72, 0xbffffab9, 0x4002fa10 <unfinished ...>
> SYS_write(2, "find: ", 6find: )                        = 6
> SYS_write(2, ".", 1.)                             = 1
> SYS_write(2, ": Value too large for defined da"..., 39: Value too large
> for defined data type) = 39
> SYS_write(2, "\n", 1
> )                            = 1
> <... error resumed> )                            = 0
> fchdir(3 <unfinished ...>
> SYS_fchdir(3, 0, 0, 0, 0xbffffab9)               = 0
> <... fchdir resumed> )                           = 0
> strchr("-!(),", '-')                             = "-!(),"
> exit(1 <unfinished ...>
> SYS_exit_group(1)                                = <void>
> SYS_exit(1 <unfinished ...>
> +++ exited (status 1) +++
> poppins[reida]:


Anonymous
Mon 05 Apr 2004 04:31:17 PM UTC, comment #1: 

The likely difference between the two versions of find is that they use a different version of gnulib, and the updated version is now sensitive to this problem.  Alternatively, maybe it's a glibc difference.  Certainly the differences between the find application code is minimal.

Looking at the ltrace output, it looks like readdir() is failing.  The likely bit of affected code is gnulib's savedir.c.  That certainly should faiil if readdir() fails.

I suspect that somewhere in there is a filename whose base
name is more than 256 characters.   To find out, please try this (on either the client or the server):

ls -R1 | awk '(length > 200) {print length, $0;}' | sort -n | tail

Try picking the last filename from the above and invoking find directly on it.   The idea here is to pin things down to one path name that successfully reproduces the error.

Lastly, the output of "ltrace" isn't very helpful about the data that's returned from the system calls.   What do you get if you use "strace"?   What about "ltrace -S"?

James Youngman <jay>
Group administrator
Mon 05 Apr 2004 03:29:16 PM UTC, original submission:  

This is a copy of <http://bugs.debian.org/241538>.
Andrew Reid writes:
--------------------------
Version: 4.1.20-3

I am having a rather strange difficulty with the "find" command.

For many paths on my system, including all paths in my
account, I get:

> # find <path> -print
> find: <path>: Value too large for defined data type


The failure occurs in some directories but not others, as described
below.

The set-up of the machine is such that user home directories can be
accessed by multiple routes from the start of the file system.  I am
not certain that this is relevant, but describe it here for
completeness.
 
Users' actual home directories are divided into several numbered
"home" directories from the root, like this: /u/home<n>/<user>/

All user home directories are accessible via symlinks from
/u/users/<user> -> /u/home<n>/<user>

There is also a symlink at the root of the filesystem,

/users -> /u/users

The client is an x86 machine.  The filesystems are remotely mounted
"xfs" file systems, mounted via NFS from an SGI file server.  Each
home<n> subdirectory is separately mounted on the client by way of an
automounter.  Most of these reside on separate disks on the server,
except for home1 and home2, which share a physical disk on the server.
The problem occurs for some of the /u/home<n>/ directories, but not
others.

All of these directories are exported identically (to each other) from
the file server, and imported identically by the client.  There are
hardware differences between the mounted disks, which correlate with
the "find" problem -- home directories which occupy 48GB disks and use
more than 28GB of them seem to have this problem.  The problem is also
common to all of the largest filesystems.

On a Debian "stable" client, importing the same file systems from the
same file server, running the same kernel, but using "find" from
version 4.1.7-2 of findutils, the problem does not occur.

On all systems, these calls work.
# find /u -maxdepth 1    
# find /u/users -maxdepth 1
 
On the problem system in the problem directories, these calls:
# find /u/home<n> -maxdepth 1
fail with the "Value too large" error -- this path argument
is the direct path to the home directory.

Although the filesystem and automounter configuration on these
machines is not trivial, it is nevertheless identical between "stable"
and "testing" machines, and the "find" bug occurs only on the
"testing" machines.  There is circumstantial evidence that the problem
involves large xfs filesystems.
--------------------------

He later provided me with the ltrace I am attaching.
      thanks, cu andreas

Andreas Metzler <ametzler>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #1187:  findbug.ltrace added by ametzler (31KiB - application/octet-stream - ltrace output)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by ametzler (original report)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 7 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2005-01-03 jay Open/ClosedOpen Closed
    2004-12-05 jay StatusNone Invalid
        Assigned toNone jay
    2004-12-02 jay Carbon-Copy- Added timwaugh<twaugh --AT-- redhat --DOT-- com>
    2004-08-09 ametzler Carbon-Copy- Added 241538 --AT-- bugs --DOT-- debian --DOT-- org
    2004-04-05 ametzler Attached File- Added findbug.ltrace, #1169
        Carbon-Copy- Added andrewreid<andrew --DOT-- reid --AT-- nist --DOT-- gov>

    Back to the top

    Powered by Savane 3.13-d3ae.
    Corresponding source code