mainGNU Core Utilities - Support: sr #107875, BUG cp -u corrupts 'fs''...


You are not allowed to post comments on this tracker with your current authentication level.

sr #107875: BUG cp -u corrupts 'fs'' information if interupted; can't recover on future invoctions

Submitter:  Linda A. Walsh <law>
Submitted:  Tue 15 Nov 2011 05:58:22 PM UTC
Category:  None Priority:  5 - Normal
Severity:  3 - Normal Status:  None
Privacy:  Public Assigned to:  None
Open/Closed:  Open Operating System:  None

Tue 15 Nov 2011 05:58:22 PM UTC, original submission:  

This should be filed under bugs, not under support, but it seems that users of the core utilis are ot allowed to find bugs...convenient.  No wonder quality metrics worthless.

Not trying for a sensationalist summary, but you try coming up with a SHORT accurate summary for this.

The problem is bad (in the sense of providing false assurance and not being reliable), but not as bad as the summary might sound...

if you copy a bunch of files (or 1 file for that matter, but then it might be more quickly noticed, and the copy is interrupted (most often control-C, cuz some param was forgotten, but could be other causes),  a partial file with the current time stamp is left in the target location and the corrupt copy is not removed upon interruption, though it is marked as being "current" (w/current DT stamp).

This creates a corrupt copy of the file in a collection of files that subsequent cp -u won't correct.  This is a problem.

As there is no indication in a collection of how many files are corrupted in this manner...and the sources may have long been deleted. 

If interrupted, the cp tool should remove any partials or ensure they are not created to begin with.

Possible ways of addressing:
A) catch INT (& catchable signals), and remove any files that are 'incomplete'
Besides that, several other steps could be taken to provide increasing protections (some are orthogonal, some dependent):
B) 1). open destination name for write (verifying accesses) w/
       Exclusive Write;
   2). open tmp file for actual cp operation.
   3). use posix_fallocate (if available) to allocate sufficient space for the copy
   4). do the copy.
   5); rename tmp over original; (closing original before rename on systems that don't support separation of names and FD's (Win systems et al).
C) reset DT stamps on newly opened files to '0' (~1969/70?)' in all non-auto-updated fields; -- then start copy...  any future
invokations of "cp -u could examine the time stamps, and if the non-auto-updated fields appear to be zero; do the copy (and correct the time stamps) with 2 possible exception conditions being noted:
      (a) if the source file also has '0'd time fields, then check file sizes:
       if they match presume 'ok' (a statistical 'guess', -- possibly warned about with a -verbose option),
       if sizes don't match, presume not a correct update and do the copy. 
D) others?

As this is, it creates a situation of cp being unreliable.

Note, 'rsync' isn't a great substitute either, as I've ntoed
that when I was updating files with 'rsync', (which is always slower on full file copies) with equivalent options, a later
usage of "cp -uav to copy the files recopied most of the files
(all? not sure)  that rsync had copied with -aUVHAX (supposedly the same info as cp -au from my understanding)).

The same was not true for the reverse case (files cp'ed and updated by cp, were not updated by rsync, -- leading me to suspect rsync as not only being significantly slower, but not as thorough in copying over information).

FWIW, I feel it important to file bugs about tools that are currently the best in their class...(and tend to devote my attentions to wanting to see them enhanced, even beyond their original scope at times);  rsync used to have a very basic feature which put it above cp, ... it copied extended attrs and ACLS.  Now that cp does that, and that cp was about 2-3x faster
than rsync for full files...

Linda A. Walsh <law>


(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

No files currently attached


Depends on the following items: None found

Items that depend on this one: None found


Carbon-Copy List
  • -email is unavailable- added by law
  • -email is unavailable- added by law (Submitted the item)

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.


    Follows 1 latest change.

    Date Changed by Updated Field Previous Value => Replaced by
    2011-11-15 law Carbon-Copy- Added -email is unavailable-

    Back to the top

    Powered by Savane 3.13-3cd0.
    Corresponding source code