bugGNU Wget - Bugs: bug #20414, Wget should accept/reject on URL,...

 
 

bug #20414: Wget should accept/reject on URL, not file.

Submitted by:  Noël Köthe <nok>
Submitted on:  Mon 09 Jul 2007 08:04:52 PM UTC  
 
Category: Feature RequestSeverity: 3 - Normal
Priority: 7 - HighStatus: Invalid
Privacy: PublicAssigned to: None
Originator Name: Open/Closed: Closed
Release: trunkOperating System: GNU/Linux
Reproducibility: NoneFixed Release: None
Planned Release: 1.12Regression: None
Work Required: 0 - HoursPatch Included: Yes

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

Mon 15 Oct 2007 12:40:00 AM UTC, comment #3:

This has been closed downstream; doing likewise here.

Micah Cowan <micahcowan>
Mon 15 Oct 2007 12:31:26 AM UTC, comment #2:

We don't seem to be currently matching against .1 filenames and the like. I'm not sure what this patch is supposed to fix; have queried the downstream bug reporter. u->file should just be the tail portion of u->url (except un-percent-encoded).

Micah Cowan <micahcowan>
Tue 02 Oct 2007 11:52:05 PM UTC, comment #1:

This isn't precisely right, either (now that we have content-disposition support, that is). Clearly, such things as suffixes, and forced .html extension-adding, should not be considered for accept/reject. However, the URL would be inappropriate if a Content-Disposition header is given.

Micah Cowan <micahcowan>
Mon 09 Jul 2007 08:04:52 PM UTC, original submission:

Hello,

a forwarded bugreport with a patch from http://bugs.debian.org/294336

"I don't know whether this behaviour is intended or not: during recursive retrieval, when wget has to decide whether to enqueue or not a discovered url, it scans accept/reject lists with u->file, instead of u->url. as a result, it is often wrong about what is to be crawled.

A very short patch is included. However, I am not 100% sure it agrees with the original author's intended behaviour. For sure, recursive retrievals might change significantly, so I don't recommend applying this patch without extensive testing.

regards
g.b.

--- wget-1.9.1/src/recur.c.old 2003-10-11 15:57:11.000000000 +0200
+++ wget-1.9.1/src/recur.c 2005-02-04 18:15:44.000000000 +0100
@@ -525,7 +525,7 @@
&& depth != INFINITE_RECURSION
&& depth < opt.reclevel - 1))
{
- if (!acceptable (u->file))
+ if (!acceptable (url))
{
DEBUGP (("%s (%s) does not match acc/rej rules.\n",
url, u->file));

"

Noël Köthe <nok>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by micahcowan (Updated the item)
  • -unavailable- added by nok (Submitted the item)
  •  

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 7 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Mon 15 Oct 2007 12:40:00 AM UTCmicahcowanStatusNeeds Discussion=>Invalid
      Open/ClosedOpen=>Closed
    Thu 23 Aug 2007 01:29:25 AM UTCmicahcowanSummarywget: wrong behaviour with -A and -R during recursive retrieval=>Wget should accept/reject on URL, not file.
    Wed 01 Aug 2007 12:06:23 AM UTCmicahcowanPriority5 - Normal=>7 - High
      StatusNone=>Needs Discussion
    Fri 20 Jul 2007 11:48:54 PM UTCmicahcowanWork RequiredNone=>0 - Hours
    Fri 20 Jul 2007 05:23:38 AM UTCmicahcowanPlanned ReleaseNone=>1.12

    Back to the top


    Powered by Savane 3.1-cleanup1