bugGNU Wget - Bugs: bug #34855, Wget 1.12: first retrieves...

 
 

bug #34855: Wget 1.12: first retrieves --reject list files, then deletes, rather than just skipping

Submitted by:  Aaron Kulkis <akulkis>
Submitted on:  Fri 18 Nov 2011 02:55:41 AM UTC  
 
Category: Program LogicSeverity: 3 - Normal
Priority: 5 - NormalStatus: None
Privacy: PublicAssigned to: None
Originator Name: akulkisOpen/Closed: Open
Release: 1.12Operating System: GNU/Linux
Reproducibility: Every TimeFixed Release: None
Planned Release: NoneRegression: None
Work Required: NonePatch Included: None

Add a New Comment (Rich MarkupRich Markup):
   

You are not logged in

Please log in, so followups can be emailed to you.

 

Thu 03 May 2012 01:37:09 PM UTC, comment #1:

works as documented: "Note that these two options do not affect the downloading of html files (as determined by a ‘.htm’ or ‘.html’ filename prefix). This behavior may not be desirable for all users, and may be changed for future versions of Wget."

~mk

Anonymous
Fri 18 Nov 2011 02:55:41 AM UTC, original submission:

Excerpted from output of:
$ wget -c -r --reject=.rpm,".iso","mirrorlist","tar.gz","index.html?","buntu","Debian","ebian","CentOS","Fedora","RHEL","SLE","Redhat","Mandriva", -Xrepodata http://download.opensuse.org/repositories

START EXCERPT

asking libproxy about url 'http://download.opensuse.org/repositories/home:/MasterPatricko:/?C=N;O=A'

libproxy suggest to use 'direct://'
--2011-11-17 21:25:31-- http://download.opensuse.org/repositories/home:/MasterPatricko:/?C=N;O=A
Connecting to download.opensuse.org|195.135.221.134|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1295 (1.3K) [text/html]
Saving to: “download.opensuse.org/repositories/home:/MasterPatricko:/index.html?C=N;O=A”

100%[===========================================================================================================>] 1,295 --.-K/s in 0.001s

2011-11-17 21:25:32 (862 KB/s) - “download.opensuse.org/repositories/home:/MasterPatricko:/index.html?C=N;O=A” saved [1295/1295]

Removing download.opensuse.org/repositories/home:/MasterPatricko:/index.html?C=N;O=A since it should be rejected.

END EXCERPT
NOTE LAST LINE.

Logic within main loop appears to be

while (! done) {
determine_next(url)
download(url,localfile)
if inlist(url,rejectlist) {
remove(localfile)
prinf_file_removed_message();
}
}

SHOULD BE:

while (! done) {
determine_next(url);
if (! inlist(url,rejectlist))
download(url,localfile);
}

akulkis@kulkilap:~> wget --version
GNU Wget 1.12 built on linux-gnu.

+digest +ipv6 +nls +ntlm +opie +md5/openssl +https -gnutls +openssl
-iri

Wgetrc:
/etc/wgetrc (system)
Locale: /usr/share/locale
Compile: gcc -DHAVE_CONFIG_H -DSYSTEM_WGETRC="/etc/wgetrc"
-DLOCALEDIR="/usr/share/locale" -I. -I../lib -fomit-frame-pointer
-fmessage-length=0 -O2 -Wall -D_FORTIFY_SOURCE=2 -fstack-protector
-funwind-tables -fasynchronous-unwind-tables -g
Link: gcc -fomit-frame-pointer -fmessage-length=0 -O2 -Wall
-D_FORTIFY_SOURCE=2 -fstack-protector -funwind-tables
-fasynchronous-unwind-tables -g /usr/lib/libssl.so
/usr/lib/libcrypto.so -lproxy -ldl -lrt ftp-opie.o openssl.o
http-ntlm.o gen-md5.o ../lib/libgnu.a

Copyright © 2009 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
<http://www.gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Originally written by Hrvoje Nikšić <hniksic@xemacs.org>.
Currently maintained by Micah Cowan <micah@cowan.name>.
Please send bug reports and questions to <bug-wget@gnu.org>.
akulkis@kulkilap:~> uname -a
Linux kulkilap 2.6.37.6-0.7-desktop #1 SMP PREEMPT 2011-07-21 02:17:24 +0200 i686 i686 i386 GNU/Linux
akulkis@kulkilap:~>

Aaron Kulkis <akulkis>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by akulkis (Submitted the item)
  •  

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    No Changes Have Been Made to This Item

    Back to the top


    Powered by Savane 3.1-cleanup