bugGNU Wget - Bugs: bug #27473, Wget doesn't recurse for...

 
 

bug #27473: Wget doesn't recurse for redirect-looped pages

Submitter:  Micah Cowan <micahcowan>
Submitted:  Thu 17 Sep 2009 09:13:11 PM UTC
   
 
Category:  Program Logic Severity:  3 - Normal
Priority:  5 - Normal Status:  Confirmed
Privacy:  Public Assigned to:  None
Originator Name:  Open/Closed:  Open
Release:  trunk Operating System:  None
Reproducibility:  None Fixed Release:  None
Planned Release:  1.14 Regression:  None
Work Required:  0 - Hours Patch Included:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Thu 17 Sep 2009 09:13:11 PM UTC, original submission:  


wget -l inf -r  -k -U Mozilla http://introanthroclass.wikispaces.com/


This request results in the following series of redirects:


Location: https://session.wikispaces.com/session/auth?authToken=0ffd5707c4b7f8aebb55647eff86300b8
http://introanthroclass.wikispaces.com?responseToken=0ffd5707c4b7f8aebb55647eff86300b8
Location: http://introanthroclass.wikispaces.com/


Note that it loops back around to the originally-requested page. By this point, Wget has already blacklisted the original URL, though at that point it was only a redirect. When Wget encounters it the second time (in the same redirect sequence), descend_redirect_p returns false, because it had been blacklisted, setting "descend = false" within retr.c:retrieve_tree, and silently fails to recurse the page.

The workaround is to add --save-cookies=cookies.txt --load-cookies=cookies.txt --keep-session-cookies, and try again, so that on the second try, the server sees the cookies right off the bat, and doesn't redirect again in the first place.

Reproduced in 1.11.4-2ubuntu1 and in the latest dev sources (as of e4153d501c59).

Micah Cowan <micahcowan>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by micahcowan (Submitted the item)
  •  

    No changes have been made to this item

    Back to the top

    Powered by Savane 3.13-4448.
    Corresponding source code