bugfindutils - Bugs: bug #17877, Invalid "No such file or...

 
 

bug #17877: Invalid "No such file or directory" error on filesystem without stable inode numbers

Submitter:  None
Submitted:  Thu 28 Sep 2006 11:24:11 AM UTC
   
 
Category:  find Severity:  3 - Normal
Item Group:  Wrong result Status:  Wont Fix
Privacy:  Public Assigned to:  jay
Originator Name:  Miklos Szeredi Originator Email:  -email is unavailable-
Open/Closed:  Closed Release:  4.3.0
Fixed Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Sun 22 Jul 2007 02:15:13 PM UTC, comment #13: 

Looks like no further change is needed in findutils, so closing.

James Youngman <jay>
Group administrator
Fri 06 Oct 2006 08:39:27 AM UTC, comment #12: 


> If files are identified by the path, then you can hash the
> path. If you use a good 64-bit hash the chance of collision
> is practically zero. That's good enough.


Yes.  And this solution is actually practical on pure 64bit
archs only.  On 32bit and dual archs it's not practial, because
legacy apps (those compiled without largefile support) will
get an EOVERFLOW for stat if the inode number doesn't fit in
32 bits.  This is irrespective of whether the app actually
would use st_ino or not.

So in fact 64 bit inode numbers would break a lot of
currently installed systems.  This is not a theoretical
possibility, I've had bug reports in this area.

Any other solutions?

> I may sound a bit impatient here, but it's impatience for a
> reason. Knowledge of file serial numbers must be hardwired
> into thousands of programs. We can't change them all. If you
> have to support inodes by some inefficient means, then I'm
> afraid that's what you'll have to do. (Or, if you prefer,
> you can warn users of these file systems that they will
> break lots of programs. :-)


Currently I have report of exactly 1 (one) application,
that breaks.  That is an alpha version of find, for which
a solution is actually in the works (see gnulib-bugs ml).

So I think your impatience is rather unwarranted.

Miklos Szeredi <mszeredi>
Thu 05 Oct 2006 11:00:56 PM UTC, comment #11: 


>> if some Linux-based file systems can't provide stable
>> inode numbers, they should be fixed so that they do. It
>> shouldn't be that hard.
>
> It's not hard, it's impossible.  Take for example path
> based network filesystems (smbfs, sshfs).  For these the
> only identifier for files is the path.


If files are identified by the path, then you can hash the
path.  If you use a good 64-bit hash the chance of collision
is practically zero.  That's good enough.

I may sound a bit impatient here, but it's impatience for a
reason.  Knowledge of file serial numbers must be hardwired
into thousands of programs.  We can't change them all.  If you
have to support inodes by some inefficient means, then I'm
afraid that's what you'll have to do.  (Or, if you prefer,
you can warn users of these file systems that they will
break lots of programs.  :-)

Paul Eggert <eggert>
Wed 04 Oct 2006 06:58:27 PM UTC, comment #10: 


> if some Linux-based file systems can't provide stable inode
> numbers, they should be fixed so that they do. It shouldn't
> be that hard.


It's not hard, it's impossible.  Take for example path based
network filesystems (smbfs, sshfs).   For these the only identifier for files is the path.

How do you map the path to a unique stable integer identifier?

Miklos Szeredi <mszeredi>
Wed 04 Oct 2006 04:49:03 PM UTC, comment #9: 

Miklos Szeredi <miklos@szeredi.hu> writes:

> What the patch does is to release the inode as
> soon as there are no more references to it.


Sorry, but that's going to break lots of code.

It's not just gnulib; it's lots of other uses as well.  For
example, some shell scripts use "ls -i" to get a file's
inode number N, and "find -xdev ... -inum N" to find all
hard links to the same file.

Perhaps POSIX isn't clear enough on the subject, but the
intent was always that a file always retains the same inode
number, even if it's renamed or linked.

If some Linux-based file systems can't provide stable inode
numbers, they should be fixed so that they do.  It shouldn't
be that hard.

Paul Eggert <eggert>
Wed 04 Oct 2006 09:01:50 AM UTC, comment #8: 

Thanks for the patch.  I've tested it and it doesn't seem to make any difference.


Miklos Szeredi <mszeredi>
Tue 03 Oct 2006 10:26:03 PM UTC, comment #7: 

findutils should work on systems that are themselves POSIX-compliant but which support non-POSIX filesystems, at least where this is possible.

Please try the patch now attached to this bug on Savannah (from the suggestion by Jim Meyering) and let me know if it fixes the problem.   Thanks.

James Youngman <jay>
Group administrator
Sat 30 Sep 2006 11:53:28 AM UTC, comment #6: 

Good point about inode numbers being reused.

This means that what you can tell from comparing inode numbers (obtained from pathnames), according to POSIX, is:

  • If the inode numbers are different, the file is definitely a different file.


  • If the inode numbers are the same, the file might be the same file or it might be a different file that has been assigned the same inode number.


The ambiguity in the second case can be avoided by obtaining the first inode number from an open fd (which is then kept open) instead of a pathname.  (Or an fd could be used to obtain the second inode number, as long as it was open at the time the first inode number was obtained.)

(All of the above assumes that the device numbers are the same.)

Geoff Clare <geoffclare>
Fri 29 Sep 2006 08:05:18 PM UTC, comment #5: 

BTW, POSIX isn't all that clear about inode numbers.  Take this example:

touch x
rm x
mkfifo y

Is x and y the same file?  Obviously not.  Can they have the same inode number?  Yes, filesystems do reuse inode numbers.

So is st_ino a unique identifier for a file within a filesystem?
Yes, but only at a given instant.  Of course you may know that a file has not been deleted by holding a reference to it (file descriptor or cwd).  Otherwise inode numbers are hints at best.

Miklos Szeredi <mszeredi>
Fri 29 Sep 2006 05:57:58 PM UTC, comment #4: 

Fair enough, I see your point.

However there are filesystems which simply don't have the notion of inode numbers, or anything that would indentify a file other than it's name.  And it's not possible to map the name to a unique identifier with bounded memory requirements.

So while these filesystems may not be POSIX compliant, find should continue to support them.

Smbfs, cifs and sshfs are examples of this filesystem class.  I'm not sure about FAT.  Cygwin may appear to provide stable inode numbers for FAT with some clever heuristics, but I don't think it can do that under all conditions.

Miklos Szeredi <mszeredi>
Fri 29 Sep 2006 04:57:37 PM UTC, comment #3: 

POSIX does require a file's inode number to be stable.  To see this consider what happens if you stat() a pathname and remember the st_dev and st_ino values, then some time later you stat() the same pathname and observe that the st_ino value is different.  POSIX says "The st_ino and st_dev fields taken together uniquely identify the file within the system", therefore st_ino being different means that the pathname now refers to a different file.

Geoff Clare <geoffclare>
Thu 28 Sep 2006 02:17:15 PM UTC, comment #2: 

I've found the following about st_ino in POSIX:

In definitions:

  3.175 File Serial Number

  A per-file system unique identifier for a file.

In sys/stat.h:

  ino_t     st_ino     File serial number.

  The st_ino and st_dev fields taken together uniquely identify the file within the system.

I can't see any requirement about st_ino having to be stable, only that it has to be unique.

Can you please forward this bug report to gnulib?

Thanks

Miklos Szeredi <mszeredi>
Thu 28 Sep 2006 01:20:24 PM UTC, comment #1: 


> There are several Linux filesystems (including smbfs, fat, and
> some FUSE based ones) which cannot provide stable inode numbers
> for unreferenced files or directories.


That's the problem.  If an OS can't provide stable inode numbers, it is not POSIX compliant.  find shouldn't have to bend over backwards to work around broken file systems.  You should be reporting this bug to the file system people.  cygwin manages to provide stable inode numbers for FAT just fine.

That said, it would be nice if fts could recognize such broken file systems and work around their flaws.  But that is an issue for gnulib, not findutils, since find 4.3.0 is the first version to use gnulib's fts instead of its own homegrown traversal routines.

Eric Blake <ericb>
Group administrator
Thu 28 Sep 2006 11:24:11 AM UTC, original submission:  

fts_safe_changedir() in gnulib/lib/fts.c return ENOENT error if inode number of a directory changed.

There are several Linux filesystems (including smbfs, fat, and some FUSE based ones) which cannot provide stable inode numbers for unreferenced files or directories.

In their cases find may in rare circumstances (if the cached in-core inode is evicted from the cache) erronously fail to recurse into these directories.

This bug seems to have appeared in 4.3.0.

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #10892:  svannah-bug-17877-fix-attempt-001.patch added by jay (1KiB - text/x-diff - First attempt at a fix)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by jay (Posted a comment)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 5 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2007-07-22 jay StatusNone Wont Fix
        Assigned toNone jay
        Open/ClosedOpen Closed
    2006-10-03 jay Attached File- Added svannah-bug-17877-fix-attempt-001.patch, #10892
    2006-09-28 None Carbon-Copy- Added -unavailable-

    Back to the top

    Powered by Savane 3.13-4448.
    Corresponding source code