bugGNU Wget - Bugs: bug #22670, Wget matches accept/reject lists...

 
 

bug #22670: Wget matches accept/reject lists against local filenames to decide when to delete

Submitted by:  Micah Cowan <micahcowan>
Submitted on:  Wed 19 Mar 2008 11:11:25 PM UTC  
 
Category: Program LogicSeverity: 3 - Normal
Priority: 7 - HighStatus: Confirmed
Privacy: PublicAssigned to: None
Originator Name: Open/Closed: Open
Release: NoneOperating System: None
Reproducibility: NoneFixed Release: None
Planned Release: 1.13Regression: None
Work Required: 1 - DaysPatch Included: None

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

(Jump to the original submission Jump to the original submission)

Fri 25 Sep 2009 06:15:35 PM UTC, comment #6:

Not a patch-level fix, whether I want it to be or not.

Micah Cowan <micahcowan>
Tue 22 Sep 2009 07:59:05 PM UTC, comment #5:

The "delete-after" logic, meanwhile, should not follow the file's new, actual local filename (note that that also fails for content-disposition'd files), but should save the delete-after value based on whether the file should otherwise have been rejected, but matched the "traverse" accept/reject rules.

Micah Cowan <micahcowan>
Tue 03 Jun 2008 06:47:23 PM UTC, comment #4:

retrieval acc/rej and delete-after acc/rej is the wrong paradigm: at some point, if we support streamed processing, there may be no download of recurse-only URLs, and thus nothing to delete. We'll still want to control whether those links are traversed, regardless of whether they must first be downloaded before they can be traversed; it should be download acc/rej versus recurse (links-only mode) acc/rej.

Micah Cowan <micahcowan>
Fri 30 May 2008 08:03:59 PM UTC, comment #3:

Note that this problem currently results in different behavior between the following two commands:

Both download to a file named "foo" (Content-Disposition header based on the value of the "cd" query-string param); but the latter will delete the file afterwards (the acc/rej deletion stuff is only done in retrieve_tree, not retrieve_url).

Micah Cowan <micahcowan>
Thu 22 May 2008 06:58:54 PM UTC, comment #2:

Separating the concepts of fetch-accept and delete-after rules had been proposed, and seems like a good idea, combined with no longer automatically grabbing all .html files no matter what the acc/rej rules are.

Micah Cowan <micahcowan>
Mon 14 Apr 2008 04:08:55 AM UTC, comment #1:

To be clear: it's matched against the URL's "filename" to determine accept/reject, and against the local filename to delete it afterwards.

Micah Cowan <micahcowan>
Wed 19 Mar 2008 11:11:25 PM UTC, original submission:

It should match against the original URL. This leads to strange cases like "-E -A php" telling Wget that it should download and parse the PHP file, but afterwards deleting it because it had been renamed to "index.php.html", which doesn't match any longer.

Similarly, there can be issues when the local filename is ".html.1", renamed because of the use of -nd.

Micah Cowan <micahcowan>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by micahcowan (Submitted the item)
  •  

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 7 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Fri 02 Oct 2009 07:20:32 PM UTCmicahcowanPlanned Release1.14=>1.13
    Fri 25 Sep 2009 06:15:35 PM UTCmicahcowanPlanned Release1.12.x=>1.14
      Priority4=>7 - High
    Tue 22 Sep 2009 07:59:05 PM UTCmicahcowanPlanned Release1.14=>1.12.x
    Thu 21 Aug 2008 11:21:31 PM UTCmicahcowanPlanned Release1.12=>1.14
    Thu 22 May 2008 06:58:54 PM UTCmicahcowanPriority5 - Normal=>4
      Work Required0 - Hours=>1 - Days

    Back to the top


    Powered by Savane 3.1-cleanup1