Forwarded from http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=162283

Hi,

Wget always downloads retrieved symlinks, even when the retrieved
file has already been successfully downloaded.

I invoked wget like this,

    wget \
              --cut-dirs=2 \
              --glob=on \
              --input-file=NetBSD-1.6-urlist \
              --append-output=NetBSD-1.6-log \
              --retr-symlinks \
              --timestamping \
              --force-directories \
              --no-host-directories \
              --non-verbose \
            &

the files in question have this ftp directory listing,

    lrwxrwxrwx   1 ftpuser  ftpusers 27 Sep 12 03:35 man.tgz -> ../../../shared/ALL/man.tgz
    lrwxrwxrwx   1 ftpuser  ftpusers 28 Sep 12 03:35 misc.tgz -> ../../../shared/ALL/misc.tgz

I ran wget and these two files, {man,misc}.tgz, were downloaded
successfully, ie, the symlinks were followed, and the
linked-to-files were downloaded, in addition, during this run many
other regular, ie, not symlinked files, were also downloaded.

The first indication of a problem is that the downloaded symlinked
files were given the mtime of the symlink, rather than the mtime
of the linked-to-file,

    -rw-r--r--    1 jeff     jeff 6640184 Sep 12 03:35 man.tgz
    -rw-r--r--    1 jeff     jeff 2333002 Sep 12 03:35 misc.tgz

Then, I ran wget again, and here is where the problem occurred.
What happened is that the previously downloaded regular files were
not downloaded again, as expected, however, the two symlinked
files were downloaded again, in their entirety.

This is a tremendous waste of resources, and it negates the whole
purpose for using timestamping in the first place.

Thanks,
--
Jeff Sheinberg