bugGNU Wget - Bugs: bug #29518, CLI-option to delete URLs from...

 
 

bug #29518: CLI-option to delete URLs from input file automatically

Submitted by:  Rene Bartsch <renne>
Submitted on:  Sun 11 Apr 2010 05:56:09 PM UTC  
 
Category: Feature RequestSeverity: 1 - Wish
Priority: 5 - NormalStatus: None
Privacy: PublicAssigned to: None
Originator Name: Open/Closed: Open
Release: 1.11.4Operating System: GNU/Linux
Reproducibility: Every TimeFixed Release: None
Planned Release: NoneRegression: None
Work Required: NonePatch Included: None

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

Wed 12 Jan 2011 09:30:50 AM UTC, comment #4:

I have updated the branch and have made the code more robust. https://code.launchpad.net/~leonard-ehrenfried/wget/purge-input-file

I'd appreciate comments.

Leonard Ehrenfried <lenni>
Mon 01 Nov 2010 09:47:38 AM UTC, comment #3:

In my opinion the already existing wget features are good enough for managing/repeating failing requests. Just remove the URL from the input file after successful download. In case of a pipe you can use a log file masking the already downloaded URLs.

Rene Bartsch <renne>
Sun 31 Oct 2010 04:24:02 PM UTC, comment #2:

I have written a skeletal implementation of this feature and put my repository up on Launchpad.

https://code.launchpad.net/~leonard-ehrenfried/wget/purge-input-file

The following problems are still open:

- what happens when the input is a pipe
- what about a recursive download where a request fails

Leonard Ehrenfried <lenni>
Wed 20 Oct 2010 09:13:57 PM UTC, comment #1:

I would like to take a shot at tackling this.

Would a feature like this require prior discussion? I have already asked on the mailing list and IRC.

Leonard Ehrenfried <lenni>
Sun 11 Apr 2010 05:56:09 PM UTC, original submission:

The WGET option "--input-file=" is quite useful for multiple downloads.

But when WGET is called again, it tries to re-download (e.g. files removed from target directory, etc.). Using an option which makes WGET remove sucessful downloads from the input file, one can easily manage a download queue by appending URLs to the input-file before calling WGET again while already dowloaded URLs are removed.

Rene Bartsch <renne>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by lenni (Posted a comment)
  • -unavailable- added by gscrivano (Updated the item)
  • -unavailable- added by renne (Submitted the item)
  •  

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 2 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Thu 03 Jun 2010 12:53:17 PM UTCgscrivanoSeverity3 - Normal=>1 - Wish
    Fri 28 May 2010 08:33:18 AM UTCgscrivanoCategoryUser Interface=>Feature Request

    Back to the top


    Powered by Savane 3.1-cleanup1