bugGNU Wget - Bugs: bug #43229, --page-requisite doesn't follow...

 
 

bug #43229: --page-requisite doesn't follow external stylesheets

Submitter:  None
Submitted:  Tue 16 Sep 2014 06:01:03 AM UTC
   
 
Category:  Program Logic Severity:  3 - Normal
Priority:  5 - Normal Status:  Invalid
Privacy:  Public Assigned to:  None
Originator Name:  lilydjwg Originator Email:  -email is unavailable-
Open/Closed:  Closed Release:  None
Operating System:  GNU/Linux Reproducibility:  Every Time
Fixed Release:  None Planned Release:  None
Regression:  None Work Required:  None
Patch Included:  Yes
* Mandatory Fields

Add a New Comment Rich Markup
   

Fri 19 Sep 2014 01:51:09 PM UTC, comment #1: 

No, I'm afraid you've misunderstood the option. I'm quoting here from the Wget man page:

"
To download a single HTML page (or a handful of them, all specified on the command-line or in a -i URL
           input file) and its (or their) requisites, simply leave off -r and -l:

                   wget -p http://<site>/1.html

           Note that Wget will behave as if -r had been specified, but only that single page and its requisites will be downloaded.  Links
           from that page to external documents will not be followed.  Actually, to download a single page and all its requisites (even if
           they exist on separate websites), and make sure the lot displays properly locally, this author likes to use a few options in
           addition to -p:

                   wget -E -H -k -K -p http://<site>/<document>

           To finish off this topic, it's worth knowing that Wget's idea of an external document link is any URL specified in an "<A>"
           tag, an "<AREA>" tag, or a "<LINK>" tag other than "<LINK REL="stylesheet">".
"

As you can see, -p behaves just like -r with certain restrictions. As a result, even the --page-requisites option will NOT go to a different domain to download the requisites. If you want it to follow links to different domains, you must issue the -H parameter to span hosts.

Darshit Shah <darnir>
Group administrator
Tue 16 Sep 2014 06:01:03 AM UTC, original submission:  

I try to download a web page with --page-requisite, some external stylesheets still get skipped. I find from the debug output that it's skipped because it considers it's from a different domain. But as I understand it, wget is expected to download all those resources regardless of its domain in this case.

I'm downloading with the command:

wget -p -k -K http://docs.sqlalchemy.org/en/latest/search.html --debug

I'm using wget 1.15. (I do not see this in the Release pop-down box.)

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #32110:  page-requisite.patch added by None (599B - text/x-patch - A simple patch solving the problem for me)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by darnir (Posted a comment)
  • -email is unavailable- added by None (Submitted the item)
  •  

    Follow 3 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2014-09-19 darnir StatusNone Invalid
        Open/ClosedOpen Closed
    2014-09-16 None Attached File- Added page-requisite.patch, #32110

    Back to the top

    Powered by Savane 3.13-f8d8.
    Corresponding source code