bugGNU Wget - Bugs: bug #20519, -np requires / at end of dir,...


bug #20519: -np requires / at end of dir, otherwise it downloads hierarchy too much

Submitted by:  Noël Köthe <nok>
Submitted on:  Wed 18 Jul 2007 07:32:21 PM UTC  
Category: User InterfaceSeverity: 1 - Wish
Priority: 1 - LaterStatus: None
Privacy: PublicAssigned to: None
Originator Name: Open/Closed: Open
Release: 1.10.2Operating System: GNU/Linux
Reproducibility: NoneFixed Release: None
Planned Release: 1.15Regression: None
Work Required: 1 - DaysPatch Included: None

Add a New Comment(Rich Markup)

You are not logged in

Please log in, so followups can be emailed to you.


Mon 14 Apr 2008 11:47:45 AM UTC, comment #2:

Note that FTP URLs, as defined in the relevant RFCs, are also trailing-slash-sensitive. RFC 1738 defines the final component without a trailing slash as a file to be accessed, whereas a final slash would indicate a directory.

This is a low-pri feature request. I would happily accept patches that use the heuristic I described; however, I'm unlikely to bother implementing it myself.

Micah Cowan <micahcowan>
Wed 18 Jul 2007 08:09:28 PM UTC, comment #1:

HTTP has no distinction between directories and files; there is no way for wget to detect that http://example.com/foo/bar is a "directory" rather than a file; they're all just components. Note, too, that in the case without the final slash, wget will download it to a file named ".../foo/bar", not ".../foo/bar/index.html".

The exception is that, typically (and, specifically, in the specific example given in the debian bug you linked), a web server will automatically redirect URIs like "http://example.com/foo/bar" to "http://example.com/foo/bar/"; so I suppose that in this case wget could automatically detect that the web server treated it like a directory, and readjust its thinking accordingly.

Tentatively targeting at 1.12.

Micah Cowan <micahcowan>
Wed 18 Jul 2007 07:32:21 PM UTC, original submission:


"When I use wget -r -np http://example.com/foo/bar/ it will only download stuff in "bar" in below, but if I strip away the slash at the end, it will also download everything in "foo". I think this is against what users expect - wget should look at the final location of the URL given by the user (i.e. .../foo/bar/index.html and then take the last dir).

an example can be found at http://bugs.debian.org/337620
(its long but if you prefer to get it here I will paste it).


Noël Köthe <nok>


(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):

No files currently attached


Depends on the following items: None found

Items that depend on this one: None found


Carbon-Copy List
  • -unavailable- added by micahcowan (Posted a comment)
  • -unavailable- added by nok (Submitted the item)

    Please enter the title of George Orwell's famous dystopian book (it's a date):



    Follow 8 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Mon 14 Apr 2008 11:48:45 AM UTCmicahcowanStatusNeeds Discussion=>None
    Mon 14 Apr 2008 11:47:45 AM UTCmicahcowanSeverity3 - Normal=>1 - Wish
      Priority6=>1 - Later
    Sun 27 Jan 2008 04:09:57 AM UTCmicahcowanPlanned Release1.12=>1.15
    Wed 01 Aug 2007 04:45:35 AM UTCmicahcowanPriority5 - Normal=>6
    Fri 20 Jul 2007 11:55:07 PM UTCmicahcowanStatusNone=>Needs Discussion
      Work RequiredNone=>1 - Days
    Wed 18 Jul 2007 08:09:28 PM UTCmicahcowanPlanned ReleaseNone=>1.12

    Back to the top

    Powered by Savane 3.1-cleanup1