bugfindutils - Bugs: bug #34976, find: Failed to save working...

 
 

bug #34976: find: Failed to save working directory in order to [...]: Too many open files

Submitted by:  Andreas Metzler <ametzler>
Submitted on:  Sat Dec 3 09:15:03 2011  
 
Category: findSeverity: 4 - Important
Item Group: Wrong resultStatus: Fixed
Privacy: PublicAssigned to: James Youngman <jay>
Originator Name: Open/Closed: Closed
Release: 4.5.10Fixed Release: 4.5.12

Add a New Comment (Rich MarkupRich Markup):
   

You are not logged in

Please log in, so followups can be emailed to you.

 

(Jump to the original submission Jump to the original submission)

Fri Sep 20 22:01:44 2013, comment #8:

I've applied and pushed Bernhard's change (after modifying it to update the ChangeLog).

The applied patch is http://git.savannah.gnu.org/cgit/findutils.git/commit/?id=183115d0484816336f9c4d2a3198b5eae2ad4b92

I confirm that running the test on find before the code change was in effect made the test fail, and that re-compiling with the patch applied made it succeed.

Therefore I'm marking this bug as fixed.

James Youngman <jay>
Project AdministratorIn charge of this item.
Fri Aug 2 11:37:53 2013, comment #7:

Rebased and amended patch sent to ML (version 3):
http://lists.gnu.org/archive/html/bug-findutils/2013-08/msg00002.html

Bernhard Voelker <berny>
Project Member
Sun Feb 3 21:31:00 2013, comment #6:

Patch amended with a test case:
http://lists.gnu.org/archive/html/bug-findutils/2013-02/msg00016.html

Bernhard Voelker <berny>
Project Member
Mon Jan 7 19:42:41 2013, comment #5:

Patch sent to the ML:
http://lists.gnu.org/archive/html/bug-findutils/2013-01/msg00002.html

Bernhard Voelker <berny>
Project Member
Mon Jan 7 16:20:51 2013, comment #4:

The same issue has been seen in openSUSE 12.2 as originally reported in the forum:

https://forums.opensuse.org/english/other-forums/development/programming-scripting/481883-limit-find.html#post2514574

and documented in this bug:

https://bugzilla.novell.com/show_bug.cgi?id=796173

Downgrading to the findutils package version (4.4.2) which comes with the previous version of openSUSE (12.1) fixes the problem so this matches the experience mentioned in comment #3 and the original description.

Testing with the snippet of code in Comment #3 I can reproduce this easily on a standard openSUSE 12.2 system, so for duplicating setting one up and running those commands should be reliable (my system was not the one where the issue was originally found, so this is not a single-openSUSE-box bug). Bumping up the ulimit to 4096 prevents the problem when using the snippet of code from Comment #3, but increasing the number of files generated by that script by a factor of four (corresponding to the increase in the number of allowed open files via the ulimit command) results in the error again.

Aaron Burgemeister <dajoker>
Thu Dec 20 16:46:19 2012, comment #3:

Seems December is the month for running into this problem.

On a CentOS system with:

yum list installed | grep findutils
findutils.x86_64 1:4.4.2-6.el6 @base/$releasver

To reproduce easily, do the following:

-------------------------
cd /tmp
mkdir src
mkdir dest
for i in {1..1500}
do
touch /tmp/src/t$i
done
find /tmp/src -type f -execdir mv '{}' /tmp/dest \;
--------------------------

This should produce something like:

mv: error while loading shared libraries: libselinux.so.1: cannot open shared object file: Error 24
find: Failed to save working directory in order to run a command on `t1424': Too many open files

This assumes your ulimit isn't set artificially high:

ulimit -n
1024

I've seen "find -execdir" work with a very large number of files in other environments. E.g., it works on an ubuntu system with:

dpkg-query -l findutils

ii findutils 4.4.2-4ubuntu1 utilities for finding files--find, xargs

Hope this helps...

Scott Kalter <skalter>
Sun Dec 4 13:51:41 2011, comment #2:

Jay wrote

> Can you narrow this down to a simple test case?


A 1111 files tree does the trick:

(SID)ametzler@argenau:/tmp/findtest$ mkdir /tmp/findtest
(SID)ametzler@argenau:/tmp/findtest$ cd /tmp/findtest
(SID)ametzler@argenau:/tmp/findtest$ for a in `seq -w 0 9` A ; do for b in `seq -w 00 99` ; do mkdir -p $a/$b ; echo blubb > $a/$b/datei${a}${b} ;echo blubb > $a/datei${a} ; done ; done
(SID)ametzler@argenau:/tmp/findtest$ /tmp/find -type f -execdir bzip2 '{}' ';'
/tmp/find: `./7/74': Too many open files
/tmp/find: `./7/39': Too many open files
[...]

cu andreas

Andreas Metzler <ametzler>
Sat Dec 3 22:46:35 2011, comment #1:

I haven't been able to reproduce this problem (Andreas, have you?).

Can you narrow this down to a simple test case? With specific instructions to reproduce it?

James Youngman <jay>
Project AdministratorIn charge of this item.
Sat Dec 3 09:15:03 2011, original submission:

Hello,

this is http://bugs.debian.org/650795 submitted by Jonathan Nieder
--------------------------------------------------------------
Trying to build linux-2.6/squeeze:

| dh_install -X.svn debian/patches/* '/usr/src/kernel-patches/all/2.6.32/debian'
| install debian/bin/patch.apply 'debian/linux-patch-debian-2.6.32/usr/src/kernel-patches/all/2.6.32/apply/debian'
| install debian/templates/patch.unpatch 'debian/linux-patch-debian-2.6.32/usr/src/kernel-patches/all/2.6.32/unpatch/debian'
| find 'debian/linux-patch-debian-2.6.32/usr/src/kernel-patches/all/2.6.32/debian' ! -path '/series/' -type f -execdir bzip2 '{}' ';' -execdir chmod 644 '{}.bz2' ';'
| find: Failed to save working directory in order to run a command on `drm-i915-rephrase-pread+pwrite-bounds-checking-to-avoid-potential-overflow.patch': Too many open files
| make[2]: *** [install-patch] Erreur 1
| make[2] : on quitte le répertoire « /home/jrn/.cache/t/linux-2.6 »
| make[1]: *** [binary-indep] Erreur 2
| make[1] : on quitte le répertoire « /home/jrn/.cache/t/linux-2.6 »
| make: *** [binary-indep] Erreur 2
| dpkg-buildpackage: erreur: fakeroot debian/rules binary a produit une erreur de sortie de type 2

| $ dpkg-query -W findutils
| findutils 4.5.10-1
| $ ulimit -n
| 1024

It seems that "find -execdir" is not very frugal with file
descriptors.
[...]
Downgrading to 4.4.2-1 avoids trouble, fortunately.
----------------------------------------------------------

Andreas Metzler <ametzler>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

Attached Files
file #28741:  bug34976.execdir-v3.patch added by berny (8kB - text/x-patch)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by berny (Posted a comment)
  • -unavailable- added by dajoker (Posted a comment)
  • -unavailable- added by skalter (Posted a comment)
  • -unavailable- added by jay (Posted a comment)
  • -unavailable- added by jay
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 10 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Tue Sep 24 22:07:30 2013jayOpen/ClosedOpen=>Closed
      Fixed ReleaseNone=>4.5.12
    Fri Sep 20 22:01:44 2013jaySeverity3 - Normal=>4 - Important
      Item GroupNone=>Wrong result
      StatusNone=>Fixed
    Fri Aug 2 10:27:56 2013bernyAttached File-=>Added bug34976.execdir-v3.patch, #28741
    Sun Dec 4 13:51:51 2011ametzlerCarbon-CopyRemoved 20807=>-
    Sat Dec 3 22:46:35 2011jayAssigned toNone=>jay
    Sat Dec 3 22:46:22 2011jayCarbon-Copy-=>Added jonathan nieder
    Sat Dec 3 09:15:19 2011ametzlerCarbon-CopyRemoved 20807=>-

    Back to the top


    Powered by Savane 3.1-cleanup