bugGNU Wget - Bugs: bug #20519, -np requires / at end of dir,...


bug #20519: -np requires / at end of dir, otherwise it downloads hierarchy too much

Submitted by:  Noël Köthe <nok>
Submitted on:  Wed Jul 18 19:32:21 2007  
Category: User InterfaceSeverity: 1 - Wish
Priority: 1 - LaterStatus: None
Privacy: PublicAssigned to: None
Originator Name: Open/Closed: Open
Release: 1.10.2Operating System: GNU/Linux
Reproducibility: NoneFixed Release: None
Planned Release: 1.15Regression: None
Work Required: 1 - DaysPatch Included: None

Add a New Comment (Rich MarkupRich Markup):

You are not logged in

Please log in, so followups can be emailed to you.


Mon Apr 14 11:47:45 2008, comment #2:

Note that FTP URLs, as defined in the relevant RFCs, are also trailing-slash-sensitive. RFC 1738 defines the final component without a trailing slash as a file to be accessed, whereas a final slash would indicate a directory.

This is a low-pri feature request. I would happily accept patches that use the heuristic I described; however, I'm unlikely to bother implementing it myself.

Micah Cowan <micahcowan>
Project Administrator
Wed Jul 18 20:09:28 2007, comment #1:

HTTP has no distinction between directories and files; there is no way for wget to detect that http://example.com/foo/bar is a "directory" rather than a file; they're all just components. Note, too, that in the case without the final slash, wget will download it to a file named ".../foo/bar", not ".../foo/bar/index.html".

The exception is that, typically (and, specifically, in the specific example given in the debian bug you linked), a web server will automatically redirect URIs like "http://example.com/foo/bar" to "http://example.com/foo/bar/"; so I suppose that in this case wget could automatically detect that the web server treated it like a directory, and readjust its thinking accordingly.

Tentatively targeting at 1.12.

Micah Cowan <micahcowan>
Project Administrator
Wed Jul 18 19:32:21 2007, original submission:


"When I use wget -r -np http://example.com/foo/bar/ it will only download stuff in "bar" in below, but if I strip away the slash at the end, it will also download everything in "foo". I think this is against what users expect - wget should look at the final location of the URL given by the user (i.e. .../foo/bar/index.html and then take the last dir).

an example can be found at http://bugs.debian.org/337620
(its long but if you prefer to get it here I will paste it).


Noël Köthe <nok>


(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):

No files currently attached


Depends on the following items: None found

Items that depend on this one: None found


Carbon-Copy List
  • -unavailable- added by micahcowan (Posted a comment)
  • -unavailable- added by nok (Submitted the item)

    Please enter the title of George Orwell's famous dystopian book (it's a date):



    Follow 8 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Mon Apr 14 11:48:45 2008micahcowanStatusNeeds Discussion=>None
    Mon Apr 14 11:47:45 2008micahcowanSeverity3 - Normal=>1 - Wish
      Priority6=>1 - Later
    Sun Jan 27 04:09:57 2008micahcowanPlanned Release1.12=>1.15
    Wed Aug 1 04:45:35 2007micahcowanPriority5 - Normal=>6
    Fri Jul 20 23:55:07 2007micahcowanStatusNone=>Needs Discussion
      Work RequiredNone=>1 - Days
    Wed Jul 18 20:09:28 2007micahcowanPlanned ReleaseNone=>1.12

    Back to the top

    Powered by Savane 3.1-cleanup1