bugGNU Wget - Bugs: bug #49429, Option to update a mirror of a...


bug #49429: Option to update a mirror of a section of a web site

Submitted by:  Dale Worley <worley>
Submitted on:  Mon 24 Oct 2016 07:42:37 PM UTC  
Category: Feature RequestSeverity: 3 - Normal
Priority: 5 - NormalStatus: None
Privacy: PublicAssigned to: None
Originator Name: worleyOpen/Closed: Open
Release: 1.18Operating System: None
Reproducibility: NoneFixed Release: None
Planned Release: NoneRegression: None
Work Required: NonePatch Included: None

Add a New Comment(Rich Markup)

You are not logged in

Please log in, so followups can be emailed to you.


Mon 24 Oct 2016 07:42:37 PM UTC, original submission:

Once a copy of a web site has been made with --mirror, it would be useful to have a --mirror-update option that could be used to cause wget to check and update the mirror copy as necessary.

This is rather messy to do. Among other things, wget would need to maintain a status file in which the mapping from URLs to local files is kept, and the entries in it need to be retained forever (in case a previously-deleted URL becomes live again and some local page's converted link to that URL remains).

Given that such a status file is needed, it might as well be used to log "work yet to be done" in the update and periodically be written to disk to checkpoint the wget run.

Related tracker items are:
http://savannah.gnu.org/bugs/?49226 Add possibility to append to existing WARC file
http://savannah.gnu.org/bugs/?34415 Add option to delete local files/directories if they do not exist on the server
http://savannah.gnu.org/bugs/?33372 Quick resume for mirroring operation.
https://savannah.gnu.org/bugs/?25340 --mirror and --convert-links mixing poorly?

Dale Worley <worley>


(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):

No files currently attached


Depends on the following items: None found

Items that depend on this one: None found


Carbon-Copy List
  • -unavailable- added by worley (Submitted the item)

    Please enter the title of George Orwell's famous dystopian book (it's a date):



    No Changes Have Been Made to This Item

    Back to the top

    Powered by Savane 3.1-cleanup1