bugGNU Wget - Bugs: bug #49429, Option to update a mirror of a...


bug #49429: Option to update a mirror of a section of a web site

Submitted by:  Dale Worley <worley>
Submitted on:  Mon 24 Oct 2016 07:42:37 PM UTC  
Category:  Feature Request Severity:  3 - Normal
Priority:  5 - Normal Status:  None
Privacy:  Public Assigned to:  None
Originator Name:  worley Open/Closed:  Open
Release:  1.18 Operating System:  None
Reproducibility:  None Fixed Release:  None
Planned Release:  None Regression:  None
Work Required:  None Patch Included:  None

Add a New Comment(Rich Markup)

You are not logged in

Please log in, so followups can be emailed to you.


Mon 24 Oct 2016 07:42:37 PM UTC, original submission:

Once a copy of a web site has been made with --mirror, it would be useful to have a --mirror-update option that could be used to cause wget to check and update the mirror copy as necessary.

This is rather messy to do. Among other things, wget would need to maintain a status file in which the mapping from URLs to local files is kept, and the entries in it need to be retained forever (in case a previously-deleted URL becomes live again and some local page's converted link to that URL remains).

Given that such a status file is needed, it might as well be used to log "work yet to be done" in the update and periodically be written to disk to checkpoint the wget run.

Related tracker items are:
http://savannah.gnu.org/bugs/?49226 Add possibility to append to existing WARC file
http://savannah.gnu.org/bugs/?34415 Add option to delete local files/directories if they do not exist on the server
http://savannah.gnu.org/bugs/?33372 Quick resume for mirroring operation.
https://savannah.gnu.org/bugs/?25340 --mirror and --convert-links mixing poorly?

Dale Worley <worley>


(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:

No files currently attached


Depends on the following items: None found

Items that depend on this one: None found


Carbon-Copy List
  • -email is unavailable- added by worley (Submitted the item)



    No changes have been made to this item

    Back to the top

    Powered by Savane 3.4