bugfindutils - Bugs: bug #34976, find: Failed to save working...

 
 

bug #34976: find: Failed to save working directory in order to [...]: Too many open files

Submitted by:  Andreas Metzler <ametzler>
Submitted on:  Sat 03 Dec 2011 09:15:03 AM UTC  
 
Category: findSeverity: 4 - Important
Item Group: Wrong resultStatus: Fixed
Privacy: PublicAssigned to: James Youngman <jay>
Originator Name: Open/Closed: Closed
Release: 4.5.10Fixed Release: 4.5.12

Add a New Comment (Rich MarkupRich Markup):
   

You are not logged in

Please log in, so followups can be emailed to you.

 

(Jump to the original submission Jump to the original submission)

Fri 20 Sep 2013 10:01:44 PM UTC, comment #8:

I've applied and pushed Bernhard's change (after modifying it to update the ChangeLog).

The applied patch is http://git.savannah.gnu.org/cgit/findutils.git/commit/?id=183115d0484816336f9c4d2a3198b5eae2ad4b92

I confirm that running the test on find before the code change was in effect made the test fail, and that re-compiling with the patch applied made it succeed.

Therefore I'm marking this bug as fixed.

James Youngman <jay>
Project AdministratorIn charge of this item.
Fri 02 Aug 2013 11:37:53 AM UTC, comment #7:

Rebased and amended patch sent to ML (version 3):
http://lists.gnu.org/archive/html/bug-findutils/2013-08/msg00002.html

Bernhard Voelker <berny>
Sun 03 Feb 2013 09:31:00 PM UTC, comment #6:

Patch amended with a test case:
http://lists.gnu.org/archive/html/bug-findutils/2013-02/msg00016.html

Bernhard Voelker <berny>
Mon 07 Jan 2013 07:42:41 PM UTC, comment #5:

Patch sent to the ML:
http://lists.gnu.org/archive/html/bug-findutils/2013-01/msg00002.html

Bernhard Voelker <berny>
Mon 07 Jan 2013 04:20:51 PM UTC, comment #4:

The same issue has been seen in openSUSE 12.2 as originally reported in the forum:

https://forums.opensuse.org/english/other-forums/development/programming-scripting/481883-limit-find.html#post2514574

and documented in this bug:

https://bugzilla.novell.com/show_bug.cgi?id=796173

Downgrading to the findutils package version (4.4.2) which comes with the previous version of openSUSE (12.1) fixes the problem so this matches the experience mentioned in comment #3 and the original description.

Testing with the snippet of code in Comment #3 I can reproduce this easily on a standard openSUSE 12.2 system, so for duplicating setting one up and running those commands should be reliable (my system was not the one where the issue was originally found, so this is not a single-openSUSE-box bug). Bumping up the ulimit to 4096 prevents the problem when using the snippet of code from Comment #3, but increasing the number of files generated by that script by a factor of four (corresponding to the increase in the number of allowed open files via the ulimit command) results in the error again.

Aaron Burgemeister <dajoker>
Thu 20 Dec 2012 04:46:19 PM UTC, comment #3:

Seems December is the month for running into this problem.

On a CentOS system with:

yum list installed | grep findutils
findutils.x86_64 1:4.4.2-6.el6 @base/$releasver

To reproduce easily, do the following:

-------------------------
cd /tmp
mkdir src
mkdir dest
for i in {1..1500}
do
touch /tmp/src/t$i
done
find /tmp/src -type f -execdir mv '{}' /tmp/dest \;
--------------------------

This should produce something like:

mv: error while loading shared libraries: libselinux.so.1: cannot open shared object file: Error 24
find: Failed to save working directory in order to run a command on `t1424': Too many open files

This assumes your ulimit isn't set artificially high:

ulimit -n
1024

I've seen "find -execdir" work with a very large number of files in other environments. E.g., it works on an ubuntu system with:

dpkg-query -l findutils

ii findutils 4.4.2-4ubuntu1 utilities for finding files--find, xargs

Hope this helps...

Scott Kalter <skalter>
Sun 04 Dec 2011 01:51:41 PM UTC, comment #2:

Jay wrote

> Can you narrow this down to a simple test case?


A 1111 files tree does the trick:

(SID)ametzler@argenau:/tmp/findtest$ mkdir /tmp/findtest
(SID)ametzler@argenau:/tmp/findtest$ cd /tmp/findtest
(SID)ametzler@argenau:/tmp/findtest$ for a in `seq -w 0 9` A ; do for b in `seq -w 00 99` ; do mkdir -p $a/$b ; echo blubb > $a/$b/datei${a}${b} ;echo blubb > $a/datei${a} ; done ; done
(SID)ametzler@argenau:/tmp/findtest$ /tmp/find -type f -execdir bzip2 '{}' ';'
/tmp/find: `./7/74': Too many open files
/tmp/find: `./7/39': Too many open files
[...]

cu andreas

Andreas Metzler <ametzler>
Sat 03 Dec 2011 10:46:35 PM UTC, comment #1:

I haven't been able to reproduce this problem (Andreas, have you?).

Can you narrow this down to a simple test case? With specific instructions to reproduce it?

James Youngman <jay>
Project AdministratorIn charge of this item.
Sat 03 Dec 2011 09:15:03 AM UTC, original submission:

Hello,

this is http://bugs.debian.org/650795 submitted by Jonathan Nieder
--------------------------------------------------------------
Trying to build linux-2.6/squeeze:

| dh_install -X.svn debian/patches/* '/usr/src/kernel-patches/all/2.6.32/debian'
| install debian/bin/patch.apply 'debian/linux-patch-debian-2.6.32/usr/src/kernel-patches/all/2.6.32/apply/debian'
| install debian/templates/patch.unpatch 'debian/linux-patch-debian-2.6.32/usr/src/kernel-patches/all/2.6.32/unpatch/debian'
| find 'debian/linux-patch-debian-2.6.32/usr/src/kernel-patches/all/2.6.32/debian' ! -path '/series/' -type f -execdir bzip2 '{}' ';' -execdir chmod 644 '{}.bz2' ';'
| find: Failed to save working directory in order to run a command on `drm-i915-rephrase-pread+pwrite-bounds-checking-to-avoid-potential-overflow.patch': Too many open files
| make[2]: *** [install-patch] Erreur 1
| make[2] : on quitte le répertoire « /home/jrn/.cache/t/linux-2.6 »
| make[1]: *** [binary-indep] Erreur 2
| make[1] : on quitte le répertoire « /home/jrn/.cache/t/linux-2.6 »
| make: *** [binary-indep] Erreur 2
| dpkg-buildpackage: erreur: fakeroot debian/rules binary a produit une erreur de sortie de type 2

| $ dpkg-query -W findutils
| findutils 4.5.10-1
| $ ulimit -n
| 1024

It seems that "find -execdir" is not very frugal with file
descriptors.
[...]
Downgrading to 4.4.2-1 avoids trouble, fortunately.
----------------------------------------------------------

Andreas Metzler <ametzler>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

Attached Files
file #28741:  bug34976.execdir-v3.patch added by berny (8kB - text/x-patch)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by berny (Posted a comment)
  • -unavailable- added by dajoker (Posted a comment)
  • -unavailable- added by skalter (Posted a comment)
  • -unavailable- added by jay (Posted a comment)
  • -unavailable- added by jay
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 10 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Tue 24 Sep 2013 10:07:30 PM UTCjayOpen/ClosedOpen=>Closed
      Fixed ReleaseNone=>4.5.12
    Fri 20 Sep 2013 10:01:44 PM UTCjaySeverity3 - Normal=>4 - Important
      Item GroupNone=>Wrong result
      StatusNone=>Fixed
    Fri 02 Aug 2013 10:27:56 AM UTCbernyAttached File-=>Added bug34976.execdir-v3.patch, #28741
    Sun 04 Dec 2011 01:51:51 PM UTCametzlerCarbon-CopyRemoved 20807=>-
    Sat 03 Dec 2011 10:46:35 PM UTCjayAssigned toNone=>jay
    Sat 03 Dec 2011 10:46:22 PM UTCjayCarbon-Copy-=>Added jonathan nieder
    Sat 03 Dec 2011 09:15:19 AM UTCametzlerCarbon-CopyRemoved 20807=>-

    Back to the top


    Powered by Savane 3.1-cleanup