bugGNU Wget - Bugs: bug #29518, CLI-option to delete URLs from...

 
 

bug #29518: CLI-option to delete URLs from input file automatically

Submitter:  Rene Bartsch <renne>
Submitted:  Sun 11 Apr 2010 05:56:09 PM UTC
   
 
Category:  Feature Request Severity:  1 - Wish
Priority:  5 - Normal Status:  None
Privacy:  Public Assigned to:  None
Originator Name:  Open/Closed:  Open
Release:  1.11.4 Operating System:  GNU/Linux
Reproducibility:  Every Time Fixed Release:  None
Planned Release:  None Regression:  None
Work Required:  None Patch Included:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Wed 12 Jan 2011 09:30:50 AM UTC, comment #4: 

I have updated the branch and have made the code more robust. https://code.launchpad.net/~leonard-ehrenfried/wget/purge-input-file

I'd appreciate comments.

Leonard Ehrenfried <lenni>
Mon 01 Nov 2010 09:47:38 AM UTC, comment #3: 

In my opinion the already existing wget features are good enough for managing/repeating failing requests. Just remove the URL from the input file after successful download. In case of a pipe you can use a log file masking the already downloaded URLs.

Rene Bartsch <renne>
Sun 31 Oct 2010 04:24:02 PM UTC, comment #2: 

I have written a skeletal implementation of this feature and put my repository up on Launchpad.

https://code.launchpad.net/~leonard-ehrenfried/wget/purge-input-file

The following problems are still open:

- what happens when the input is a pipe
- what about a recursive download where a request fails

Leonard Ehrenfried <lenni>
Wed 20 Oct 2010 09:13:57 PM UTC, comment #1: 

I would like to take a shot at tackling this.

Would a feature like this require prior discussion? I have already asked on the mailing list and IRC.

Leonard Ehrenfried <lenni>
Sun 11 Apr 2010 05:56:09 PM UTC, original submission:  

The WGET option "--input-file=" is quite useful for multiple downloads.

But when WGET is called again, it tries to re-download (e.g. files removed from target directory, etc.). Using an option which makes WGET remove sucessful downloads from the input file, one can easily manage a download queue by appending URLs to the input-file before calling WGET again while already dowloaded URLs are removed.

Rene Bartsch <renne>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by lenni (Posted a comment)
  • -email is unavailable- added by gscrivano (Updated the item)
  • -email is unavailable- added by renne (Submitted the item)
  •  

    Follow 2 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2010-06-03 gscrivano Severity3 - Normal 1 - Wish
    2010-05-28 gscrivano CategoryUser Interface Feature Request

    Back to the top

    Powered by Savane 3.13-f8d8.
    Corresponding source code