bugfindutils - Bugs: bug #34976, find: Failed to save working...

 
 

bug #34976: find: Failed to save working directory in order to [...]: Too many open files

Submitter:  Andreas Metzler <ametzler>
Submitted:  Sat 03 Dec 2011 09:15:03 AM UTC
   
 
Category:  find Severity:  4 - Important
Item Group:  Wrong result Status:  Fixed
Privacy:  Public Assigned to:  jay
Originator Name:  Open/Closed:  Closed
Release:  4.5.10 Fixed Release:  4.5.12
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Fri 20 Sep 2013 10:01:44 PM UTC, comment #8: 

I've applied and pushed Bernhard's change (after modifying it to update the ChangeLog).

The applied patch is http://git.savannah.gnu.org/cgit/findutils.git/commit/?id=183115d0484816336f9c4d2a3198b5eae2ad4b92


I confirm that running the test on find before the code change was in effect made the test fail, and that re-compiling with the patch applied made it succeed.

Therefore I'm marking this bug as fixed.


James Youngman <jay>
Group administrator
Fri 02 Aug 2013 11:37:53 AM UTC, comment #7: 

Rebased and amended patch sent to ML (version 3):
http://lists.gnu.org/archive/html/bug-findutils/2013-08/msg00002.html

Bernhard Voelker <berny>
Group administrator
Sun 03 Feb 2013 09:31:00 PM UTC, comment #6: 
Bernhard Voelker <berny>
Group administrator
Mon 07 Jan 2013 07:42:41 PM UTC, comment #5: 
Bernhard Voelker <berny>
Group administrator
Mon 07 Jan 2013 04:20:51 PM UTC, comment #4: 

The same issue has been seen in openSUSE 12.2 as originally reported in the forum:

https://forums.opensuse.org/english/other-forums/development/programming-scripting/481883-limit-find.html#post2514574

and documented in this bug:

https://bugzilla.novell.com/show_bug.cgi?id=796173

Downgrading to the findutils package version (4.4.2) which comes with the previous version of openSUSE (12.1) fixes the problem so this matches the experience mentioned in comment #3 and the original description.

Testing with the snippet of code in Comment #3 I can reproduce this easily on a standard openSUSE 12.2 system, so for duplicating setting one up and running those commands should be reliable (my system was not the one where the issue was originally found, so this is not a single-openSUSE-box bug).  Bumping up the ulimit to 4096 prevents the problem when using the snippet of code from Comment #3, but increasing the number of files generated by that script by a factor of four (corresponding to the increase in the number of allowed open files via the ulimit command) results in the error again.

Aaron Burgemeister <dajoker>
Thu 20 Dec 2012 04:46:19 PM UTC, comment #3: 

Seems December is the month for running into this problem.

On a CentOS system with:

yum list installed | grep findutils
findutils.x86_64                  1:4.4.2-6.el6             @base/$releasver

To reproduce easily, do the following:

-------------------------
cd /tmp
mkdir src
mkdir dest
for i in {1..1500}
do
   touch /tmp/src/t$i
done
find /tmp/src -type f -execdir mv '{}' /tmp/dest \;
--------------------------

This should produce something like:

mv: error while loading shared libraries: libselinux.so.1: cannot open shared object file: Error 24
find: Failed to save working directory in order to run a command on `t1424': Too many open files

This assumes your ulimit isn't set artificially high:

ulimit -n
1024

I've seen "find -execdir" work with a very large number of files in other environments.  E.g., it works on an ubuntu system with:

 dpkg-query -l findutils

ii  findutils            4.4.2-4ubuntu1       utilities for finding files--find, xargs

Hope this helps...

Scott Kalter <skalter>
Sun 04 Dec 2011 01:51:41 PM UTC, comment #2: 

Jay wrote

> Can you narrow this down to a simple test case?


A 1111 files tree does the trick:

(SID)ametzler@argenau:/tmp/findtest$ mkdir /tmp/findtest
(SID)ametzler@argenau:/tmp/findtest$ cd /tmp/findtest
(SID)ametzler@argenau:/tmp/findtest$ for a in `seq -w 0 9` A ; do for b in `seq -w 00 99` ; do mkdir -p $a/$b ; echo blubb > $a/$b/datei${a}${b} ;echo blubb > $a/datei${a} ; done ; done
(SID)ametzler@argenau:/tmp/findtest$ /tmp/find -type f -execdir bzip2 '{}' ';'
/tmp/find: `./7/74': Too many open files
/tmp/find: `./7/39': Too many open files
[...]

cu andreas

Andreas Metzler <ametzler>
Sat 03 Dec 2011 10:46:35 PM UTC, comment #1: 

I haven't been able to reproduce this problem (Andreas, have you?).

Can you narrow this down to a simple test case?   With specific instructions to reproduce it?

James Youngman <jay>
Group administrator
Sat 03 Dec 2011 09:15:03 AM UTC, original submission:  

Hello,

this is http://bugs.debian.org/650795 submitted by Jonathan Nieder
--------------------------------------------------------------
Trying to build linux-2.6/squeeze:

| dh_install -X.svn debian/patches/* '/usr/src/kernel-patches/all/2.6.32/debian'
| install debian/bin/patch.apply 'debian/linux-patch-debian-2.6.32/usr/src/kernel-patches/all/2.6.32/apply/debian'
| install debian/templates/patch.unpatch 'debian/linux-patch-debian-2.6.32/usr/src/kernel-patches/all/2.6.32/unpatch/debian'
| find 'debian/linux-patch-debian-2.6.32/usr/src/kernel-patches/all/2.6.32/debian' ! -path '*/series/*' -type f -execdir bzip2 '{}' ';' -execdir chmod 644 '{}.bz2' ';'
| find: Failed to save working directory in order to run a command on `drm-i915-rephrase-pread+pwrite-bounds-checking-to-avoid-potential-overflow.patch': Too many open files
| make[2]: * [install-patch] Erreur 1
| make[2] : on quitte le répertoire « /home/jrn/.cache/t/linux-2.6 »
| make[1]: * [binary-indep] Erreur 2
| make[1] : on quitte le répertoire « /home/jrn/.cache/t/linux-2.6 »
| make: * [binary-indep] Erreur 2
| dpkg-buildpackage: erreur: fakeroot debian/rules binary a produit une erreur de sortie de type 2

| $ dpkg-query -W findutils
| findutils     4.5.10-1
| $ ulimit -n
| 1024

It seems that "find -execdir" is not very frugal with file
descriptors.
[...]
Downgrading to 4.4.2-1 avoids trouble, fortunately.
----------------------------------------------------------

Andreas Metzler <ametzler>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #28741:  bug34976.execdir-v3.patch added by berny (8KiB - text/x-patch)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by berny (Posted a comment)
  • -email is unavailable- added by dajoker (Posted a comment)
  • -email is unavailable- added by skalter (Posted a comment)
  • -email is unavailable- added by jay (Posted a comment)
  • -email is unavailable- added by jay
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 10 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2013-09-24 jay Open/ClosedOpen Closed
        Fixed ReleaseNone 4.5.12
    2013-09-20 jay Severity3 - Normal 4 - Important
        Item GroupNone Wrong result
        StatusNone Fixed
    2013-08-02 berny Attached File- Added bug34976.execdir-v3.patch, #28741
    2011-12-04 ametzler Carbon-CopyRemoved 20807 -
    2011-12-03 jay Assigned toNone jay
    2011-12-03 jay Carbon-Copy- Added jonathan nieder <jrnieder@gmail.com>
    2011-12-03 ametzler Carbon-CopyRemoved 20807 -

    Back to the top

    Powered by Savane 3.13-3230.
    Corresponding source code