bugGNU Wget - Bugs: bug #20422, Wget should handle IRIs.

 
 

bug #20422: Wget should handle IRIs.

Submitter:  Noël Köthe <nok>
Submitted:  Mon 09 Jul 2007 09:52:21 PM UTC
   
 
Category:  Protocol Issue Severity:  3 - Normal
Priority:  9 - Immediate Status:  Fixed
Privacy:  Public Assigned to:  micahcowan
Originator Name:  Open/Closed:  Closed
Release:  1.10.2 Operating System:  GNU/Linux
Reproducibility:  Every Time Fixed Release:  1.12
Planned Release:  1.12 Regression:  None
Work Required:  2 - Weeks Patch Included:  No
* Mandatory Fields

Add a New Comment Rich Markup
   

Thu 02 Jul 2009 05:46:03 AM UTC, comment #5: 

Merged in 139a47efb8c5

Micah Cowan <micahcowan>
Tue 22 Jan 2008 11:21:39 PM UTC, comment #4: 

Bumping down the planned release from 2.0 to 1.12 (tentatively). Some international ccTLDs, including one in Russian intended for domain names in all-Cyrillic, are expected to arrive this year. This will quickly become an in-demand feature.

Micah Cowan <micahcowan>
Mon 05 Nov 2007 08:05:16 PM UTC, comment #3: 

See RFC 3987, and also http://www.w3.org/TR/REC-html40/appendix/notes.html#non-ascii-chars

Probably, we should carry sufficient information around with the URL that we can try each of the following in turn, until we find one that works:

  1. Transcode from the document's encoding to UTF-8, and percent-encode
  2. Directly percent-encode the actual bytes that were used


There might also be cases where we'd want to try transcoding into ISO-8859-1.

Host-names, of course, should only support punycoding. Perhaps if an internationalized host name is detected, we should forgo the "actual bytes" version, as there really can't be any expected meaning for a URI that has non-ASCII characters in the host name, and non-UTF-8 characters in the remainder.

Micah Cowan <micahcowan>
Wed 22 Aug 2007 04:49:50 AM UTC, comment #2: 

cf bug 20863, "Encode URL to UTF-8 before escaping".

Micah Cowan <micahcowan>
Wed 22 Aug 2007 04:25:49 AM UTC, comment #1: 

This is really part of a larger issue, in that wget doesn't handle IRIs in general. In order to do this properly, it'll need to do things like, remember what encoding the HTML file was served as (including when wget is invoked with -c at some point after the HTML file was parsed).

Micah Cowan <micahcowan>
Mon 09 Jul 2007 09:52:21 PM UTC, original submission:  

Hello,

wget doesn't support IDN domains:

$ LC_ALL=C wget 'http://www.köln.de/'
--23:49:06--  http://www.k%C3%B6ln.de/
           => `index.html'
Resolving www.k\303\266ln.de... failed: Name or service not known.

Translating the URL to the xn-- it works.

Maybe its possible to use GNU libidn http://www.gnu.org/software/libidn/

thx.

-email is unavailable-

Noël Köthe <nok>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by sxav (Updated the item)
  • -email is unavailable- added by micahcowan (Updated the item)
  • -email is unavailable- added by nok (Submitted the item)
  •  

    Follow 14 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2009-07-02 micahcowan StatusReady For Test Fixed
        Open/ClosedOpen Closed
        Fixed ReleaseNone 1.12
    2008-08-22 micahcowan Priority6 9 - Immediate
        Assigned tosxav micahcowan
    2008-08-15 sxav StatusConfirmed Ready For Test
    2008-04-14 micahcowan Assigned toNone sxav
    2008-01-22 micahcowan Priority5 - Normal 6
    2008-01-22 micahcowan Planned Release2.0 1.12
    2007-08-22 micahcowan Summarywget doesn\'t handle IDN domains Wget should handle IRIs.
    2007-07-20 micahcowan CategoryFeature Request Protocol Issue
        Planned ReleaseNone 2.0
    2007-07-09 micahcowan StatusNone Confirmed
        Work RequiredNone 2 - Weeks

    Back to the top

    Powered by Savane 3.13-d3ae.
    Corresponding source code