bugGNU Wget - Bugs: bug #34371, recursive HTTP download doesn't...

 
 

bug #34371: recursive HTTP download doesn't obey -A in css urls

Submitted by:  None
Submitted on:  Fri 23 Sep 2011 09:25:14 PM UTC  
 
Category: Program LogicSeverity: 3 - Normal
Priority: 5 - NormalStatus: None
Privacy: PublicAssigned to: None
Originator Name: Kim ScarboroughOriginator Email: -unavailable-
Open/Closed: OpenRelease: 1.13.4
Operating System: GNU/LinuxReproducibility: Every Time
Fixed Release: NonePlanned Release: None
Regression: NoneWork Required: None
Patch Included: No

Add a New Comment (Rich MarkupRich Markup):
   

You are not logged in

Please log in, so followups can be emailed to you.

 

Fri 23 Sep 2011 09:25:14 PM UTC, original submission:

When I specify -r to a URL download, it downloads all the files, including files specified in the CSS stylesheet. If I limit it with -A, however, it ignores the CSS-specified files, even though they are of the type specified.

Here's my example session:

turkey: /home/kjs/tmp/wget> wget --version
GNU Wget 1.13.4 built on linux-gnu.

+digest +https -ipv6 -iri +large-file -nls +ntlm +opie +ssl/openssl

Wgetrc:
/etc/wgetrc (system)
Compile: gcc -DHAVE_CONFIG_H -DSYSTEM_WGETRC="/etc/wgetrc"
-DLOCALEDIR="/share/locale" -I. -I../lib -I../lib -I/include -O2
-Wall
Link: gcc -O2 -Wall /lib/libssl.so /lib/libcrypto.so /lib/libz.so -Wl,-rpath
-Wl,/lib -ldl -lz -lrt ftp-opie.o openssl.o http-ntlm.o
../lib/libgnu.a

Copyright (C) 2009 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
<http://www.gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Originally written by Hrvoje Niksic <hniksic@xemacs.org>.
Please send bug reports and questions to <bug-wget@gnu.org>.
turkey: /home/kjs/tmp/wget> wget -r -np -nd -nH http://www.unknown.nu/misc/wget-testcase/test.html
--2011-09-23 16:15:22-- http://www.unknown.nu/misc/wget-testcase/test.html
Resolving http://www.unknown.nu... 66.90.79.99
Connecting to http://www.unknown.nu|66.90.79.99|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 224 [text/html]
Saving to: `test.html'

100%[======================================>] 224 --.-K/s in 0s

2011-09-23 16:15:22 (12.4 MB/s) - `test.html' saved [224/224]

Loading robots.txt; please ignore errors.
--2011-09-23 16:15:22-- http://www.unknown.nu/robots.txt
Reusing existing connection to http://www.unknown.nu:80.
HTTP request sent, awaiting response... 200 OK
Length: 95 [text/plain]
Saving to: `robots.txt'

100%[======================================>] 95 --.-K/s in 0s

2011-09-23 16:15:22 (8.67 MB/s) - `robots.txt' saved [95/95]

--2011-09-23 16:15:22-- http://www.unknown.nu/misc/wget-testcase/test.css
Reusing existing connection to http://www.unknown.nu:80.
HTTP request sent, awaiting response... 200 OK
Length: 41 [text/css]
Saving to: `test.css'

100%[======================================>] 41 --.-K/s in 0s

2011-09-23 16:15:22 (3.82 MB/s) - `test.css' saved [41/41]

--2011-09-23 16:15:22-- http://www.unknown.nu/misc/wget-testcase/html.gif
Reusing existing connection to http://www.unknown.nu:80.
HTTP request sent, awaiting response... 200 OK
Length: 1326 (1.3K) [image/gif]
Saving to: `html.gif'

100%[======================================>] 1,326 --.-K/s in 0s

2011-09-23 16:15:22 (70.2 MB/s) - `html.gif' saved [1326/1326]

--2011-09-23 16:15:22-- http://www.unknown.nu/misc/wget-testcase/css.gif
Reusing existing connection to http://www.unknown.nu:80.
HTTP request sent, awaiting response... 200 OK
Length: 1326 (1.3K) [image/gif]
Saving to: `css.gif'

100%[======================================>] 1,326 --.-K/s in 0s

2011-09-23 16:15:22 (70.5 MB/s) - `css.gif' saved [1326/1326]

FINISHED --2011-09-23 16:15:22--
Total wall clock time: 0.02s
Downloaded: 5 files, 2.9K in 0s (38.9 MB/s)
turkey: /home/kjs/tmp/wget> ls
css.gif html.gif robots.txt test.css test.html
turkey: /home/kjs/tmp/wget> rm *
turkey: /home/kjs/tmp/wget> wget -r -np -nd -nH -A gif http://www.unknown.nu/misc/wget-testcase/test.html
--2011-09-23 16:15:35-- http://www.unknown.nu/misc/wget-testcase/test.html
Resolving http://www.unknown.nu... 66.90.79.99
Connecting to http://www.unknown.nu|66.90.79.99|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 224 [text/html]
Saving to: `test.html'

100%[======================================>] 224 --.-K/s in 0s

2011-09-23 16:15:35 (18.6 MB/s) - `test.html' saved [224/224]

Loading robots.txt; please ignore errors.
--2011-09-23 16:15:35-- http://www.unknown.nu/robots.txt
Reusing existing connection to http://www.unknown.nu:80.
HTTP request sent, awaiting response... 200 OK
Length: 95 [text/plain]
Saving to: `robots.txt'

100%[======================================>] 95 --.-K/s in 0s

2011-09-23 16:15:35 (8.64 MB/s) - `robots.txt' saved [95/95]

Removing test.html since it should be rejected.

--2011-09-23 16:15:35-- http://www.unknown.nu/misc/wget-testcase/html.gif
Reusing existing connection to http://www.unknown.nu:80.
HTTP request sent, awaiting response... 200 OK
Length: 1326 (1.3K) [image/gif]
Saving to: `html.gif'

100%[======================================>] 1,326 --.-K/s in 0s

2011-09-23 16:15:35 (104 MB/s) - `html.gif' saved [1326/1326]

FINISHED --2011-09-23 16:15:35--
Total wall clock time: 0.01s
Downloaded: 3 files, 1.6K in 0s (45.9 MB/s)
turkey: /home/kjs/tmp/wget> ls
html.gif robots.txt

Note that the second time it only downloaded html.gif, which is specified in the html, but not css.gif (specified in css.gif).

I'll keep that URL live but I'm attaching the html & css files just in case.

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

Attached Files
file #24019:  test.css added by None (41B - text/css)
file #24020:  test.html added by None (224B - text/html)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by None (Submitted the item)
  •  

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 2 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Fri 23 Sep 2011 09:25:14 PM UTCNoneAttached File-=>Added test.css, #24019
      Attached File-=>Added test.html, #24020

    Back to the top


    Powered by Savane 3.1-cleanup