bugGNU Wget - Bugs: bug #20454, Added new option --exclude-files

 
 

bug #20454: Added new option --exclude-files

Submitted by:  None
Submitted on:  Thu 12 Jul 2007 04:36:31 AM UTC  
 
Category: Feature RequestSeverity: 3 - Normal
Priority: 5 - NormalStatus: Wont Fix
Privacy: PublicAssigned to: None
Originator Name: Joshua David WilliamsOriginator Email: -unavailable-
Open/Closed: ClosedRelease: None
Operating System: GNU/LinuxReproducibility: None
Fixed Release: NonePlanned Release: None
Regression: NoneWork Required: None
Patch Included: Yes

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

Fri 17 Aug 2007 10:46:09 PM UTC, comment #5:

I've submitted bug 20808 to address that problem.

Micah Cowan <micahcowan>
Fri 17 Aug 2007 10:11:27 PM UTC, comment #4:

I am using wget on a Fedora Core 6 box, and try to mirror a tikiwiki site.

#!/bin/sh

URL=https://www.someremotesite.com/prefix/twiki/bin/view/Main/

INCLUDE="/prefix/twiki/bin/viewauth/Main,/prefix/twiki/bin/view/Main,/prefix/twiki/pub"
EXCLUDE="/prefix/twiki/bin/*"
REJECT="\\?rev,\\?sortcol,\\?raw,\\?skin,\\?template"
wget --no-check-certificate -erobots=off \
--user=myuser --password=mypass \
-I $INCLUDE -X $EXCLUDE -R $REJECT -r $URL -k

Here is part of the log:
...
--15:05:48-- https://www.someremotesite.com/prefix/twiki/bin/view/Main/G1b3Perf
ormanceResults?template=viewprint&amp;rev=2&amp;sortcol=0;table=15;up=0
Reusing existing connection to http://www.someremotesite.com:443.
HTTP request sent, awaiting response... 200 OK
Length: 65377 (64K) [text/html]
Saving to: `www.someremotesite.com/prefix/twiki/bin/view/Main/G1b3PerformanceRes
ults?template=viewprint&amp;rev=2&amp;sortcol=0;table=15;up=0'

0K .......... .......... .......... .......... .......... 78% 52.5K 0s
50K .......... ... 100% 228K=1.0s

15:05:50 (63.0 KB/s) - `www.someremotesite.com/prefix/twiki/bin/view/Main/G1b3Pe
rformanceResults?template=viewprint&amp;rev=2&amp;sortcol=0;table=15;up=0' saved
[65377/65377]

Removing http://www.someremotesite.com/prefix/twiki/bin/view/Main/G1b3PerformanceResult
s?template=viewprint&amp;rev=2&amp;sortcol=0;table=15;up=0 since it should be rejected.
...

Frank Liu <liug>
Fri 17 Aug 2007 09:30:47 PM UTC, comment #3:

Well, if that is indeed the case (could you submit an example for verification?), I would think the answer should be to "fix" the -R option, rather than introduce a whole new option.

Micah Cowan <micahcowan>
Fri 17 Aug 2007 09:11:30 PM UTC, comment #2:

I don't agree with you Micah.
With the reject option, wget still downloads the files, though not saves. Last time when I tried to mirror a site and used "reject" to exclude files (90% of the site), wget still downloaded everything (though not saved on local disk) and the site admin got very upset about it. I thought I only downloaded the 10% of content that I needed, but wget actually pulled all the content.

Hopefully, with the real exclude option, wget will just skip and won't even pull those files that match the pattern.

Frank Liu <liug>
Fri 20 Jul 2007 07:02:55 AM UTC, comment #1:

As has been pointed out on the list, the -R option already provides this functionality, when wildcards are used.

This whole area of rejects/includes needs to be re-thought, but that can't really happen at this time.

I'm going to close this bug out, as the requested behavior already appears to be available.

Micah Cowan <micahcowan>
Thu 12 Jul 2007 04:36:31 AM UTC, original submission:

The attached patch is a new option to exclude specific (including wildchars) files from recursive downloads, as opposed to directories only.

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

Attached Files
file #13294:  patch.tar.bz2 added by None (2KiB - application/x-bzip - Included tarball of the patch from svn diff)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by liug (Posted a comment)
  • -unavailable- added by micahcowan (Posted a comment)
  •  

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 3 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Fri 20 Jul 2007 07:02:55 AM UTCmicahcowanStatusNone=>Wont Fix
      Open/ClosedOpen=>Closed
    Thu 12 Jul 2007 04:36:31 AM UTCNoneAttached File-=>Added patch.tar.bz2, #13294

    Back to the top


    Powered by Savane 3.1-cleanup1