bugGNU Wget - Bugs: bug #49429, Option to update a mirror of a...


bug #49429: Option to update a mirror of a section of a web site

Submitter:  Dale Worley <worley>
Submitted:  Mon 24 Oct 2016 07:42:37 PM UTC
Category:  Feature Request Severity:  3 - Normal
Priority:  5 - Normal Status:  None
Privacy:  Public Assigned to:  None
Originator Name:  worley Open/Closed:  Open
Release:  1.18 Operating System:  None
Reproducibility:  None Fixed Release:  None
Planned Release:  None Regression:  None
Work Required:  None Patch Included:  None
* Mandatory Fields

Add a New Comment Rich Markup

Mon 24 Oct 2016 07:42:37 PM UTC, original submission:  

Once a copy of a web site has been made with --mirror, it would be useful to have a --mirror-update option that could be used to cause wget to check and update the mirror copy as necessary.

This is rather messy to do.  Among other things, wget would need to maintain a status file in which the mapping from URLs to local files is kept, and the entries in it need to be retained forever (in case a previously-deleted URL becomes live again and some local page's converted link to that URL remains).

Given that such a status file is needed, it might as well be used to log "work yet to be done" in the update and periodically be written to disk to checkpoint the wget run.

Related tracker items are:
http://savannah.gnu.org/bugs/?49226  Add possibility to append to existing WARC file
http://savannah.gnu.org/bugs/?34415  Add option to delete local files/directories if they do not exist on the server
http://savannah.gnu.org/bugs/?33372  Quick resume for mirroring operation.
https://savannah.gnu.org/bugs/?25340  --mirror and --convert-links mixing poorly?

Dale Worley <worley>


(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:

No files currently attached


Depends on the following items: None found

Items that depend on this one: None found


Carbon-Copy List
  • -email is unavailable- added by worley (Submitted the item)

    No changes have been made to this item

    Back to the top

    Powered by Savane 3.12.
    Corresponding source code