bugGNU arch -- a revision control system - Bugs: bug #7088, inode-sigs in revision libraries...


bug #7088: inode-sigs in revision libraries is fundamentally broken for corruption detection

Submitter:  None
Submitted:  Mon 05 Jan 2004 08:14:48 AM UTC
Category:  tla Severity:  3 - Normal
Item Group:  bug Status:  None
Privacy:  Public Open/Closed:  Open
Release:  1.1
Fixed Release: 
Merge Request?:  None
Your Archive Name: 
Your Archive Location: 
Assigned to:  None
* Mandatory Fields

Add a New Comment Rich Markup

Mon 09 Feb 2004 12:11:43 AM UTC, comment #2: 

the above solution looks quite reasonable, and NFS compatible, it also shouldn't break when the admin does a dump/restore of the filesystem.

what is "File ID" referring to though? if its inode i would drop that as it will change on a fs dump/restore operation.

Fri 06 Feb 2004 11:50:44 PM UTC, comment #1: 

I would agree that using MD5 for the sigs would make more sense than raw inode information.

Using MD5 would also give a good opportunity to further optimize the library by linking duplicates.

I would propose the sigs store the following information:

File ID, size, mtime, MD5 hash, then use the follwing criterias when verifying library integrity

1. If size differs then library corrupt.

2. If mtime is correct then let normal operations assume the file is correct.

3. If mtime differs then calculate the MD5. If MD5 different library corrupt. If the resulting MD5 is correct then update the ,,inode-sigs index to avoid having to recalculate the MD5 again.

A nice extension of this would be to write a "library-verify" operation which verifies the MD5 of each file in a library revision (or whole library tree) and alerts on any corruption detected.

This would also make it quite trivial to write a "library-optimize" function which reads the inode sigs of existing library revisions and links identical files to optimize disk usage beyond what the normal linking process while building the library revisions can do.

Henrik Nordstrom <hno>
Mon 05 Jan 2004 08:14:48 AM UTC, original submission:  

tla 1.1 started using inode-sigs in revision libraries to detect corruption of them.

this is fundamentally broken for the following reasons:

1) its inherently incompatible with NFS setups.  I have /home NFS mounted to several boxes of differing architectures, I tend to do most development on the NFS server, but must be able to do checkouts, and such from NFS clients (due to architectural dependencies of some of my
programs) this is now impossible since tla declares my revision libraries `corrupt' because the dev field in the inode is different on an NFS mount then it is on the server.  This behavior renders arch nearly unusable for multi-machine sets sharing /home via NFS. 

2) this will prevent a system administrator from performing any kind of backup/restore operation since even when he takes care to preserve and restore everything (hard links timestamps etc), it is not possible to restore inode numbers, thus tla will declare the entire restored
revision library as `corrupt' even though its not.   similar results will occur if the admin rearranges disks or partitions in any way which results in the dev field of the inode changing.  It is not reasonable to prevent the sysadmin from using his normal tools to do backup and restore.

the inode-sigs system may be all fine and good for an optimization for diffing checked out project trees (since worse case tla just has to diff the entire tree the normal way), but its clearly unacceptable for integrity checking a revision library, its too fragile and fundamentally incompatible with NFS setups.  integrity checking of a
revision library should be done with checksums, either md5 or something more lightweight, or else not at all, at a MINIMUM there would have to be a way to permanently disable this flawed method.



(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:

No files currently attached


Depends on the following items: None found

Items that depend on this one: None found


CC list is empty


There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

Only logged-in users can vote.


Follows 1 latest change.

Date Changed by Updated Field Previous Value => Replaced by
2004-02-06 hno Carbon-Copy- Added hno

Back to the top

Powered by Savane 3.13-3cd0.
Corresponding source code