bugGNU Wget - Bugs: bug #20422, Wget should handle IRIs.

 
 

bug #20422: Wget should handle IRIs.

Submitted by:  Noël Köthe <nok>
Submitted on:  Mon 09 Jul 2007 09:52:21 PM UTC  
 
Category: Protocol IssueSeverity: 3 - Normal
Priority: 9 - ImmediateStatus: Fixed
Privacy: PublicAssigned to: Micah Cowan <micahcowan>
Originator Name: Open/Closed: Closed
Release: 1.10.2Operating System: GNU/Linux
Reproducibility: Every TimeFixed Release: 1.12
Planned Release: 1.12Regression: None
Work Required: 2 - WeeksPatch Included: No

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

Thu 02 Jul 2009 05:46:03 AM UTC, comment #5:

Merged in 139a47efb8c5

Micah Cowan <micahcowan>In charge of this item.
Tue 22 Jan 2008 11:21:39 PM UTC, comment #4:

Bumping down the planned release from 2.0 to 1.12 (tentatively). Some international ccTLDs, including one in Russian intended for domain names in all-Cyrillic, are expected to arrive this year. This will quickly become an in-demand feature.

Micah Cowan <micahcowan>In charge of this item.
Mon 05 Nov 2007 08:05:16 PM UTC, comment #3:

See RFC 3987, and also http://www.w3.org/TR/REC-html40/appendix/notes.html#non-ascii-chars

Probably, we should carry sufficient information around with the URL that we can try each of the following in turn, until we find one that works:

  1. Transcode from the document's encoding to UTF-8, and percent-encode
  2. Directly percent-encode the actual bytes that were used

There might also be cases where we'd want to try transcoding into ISO-8859-1.

Host-names, of course, should only support punycoding. Perhaps if an internationalized host name is detected, we should forgo the "actual bytes" version, as there really can't be any expected meaning for a URI that has non-ASCII characters in the host name, and non-UTF-8 characters in the remainder.

Micah Cowan <micahcowan>In charge of this item.
Wed 22 Aug 2007 04:49:50 AM UTC, comment #2:

cf bug 20863, "Encode URL to UTF-8 before escaping".

Micah Cowan <micahcowan>In charge of this item.
Wed 22 Aug 2007 04:25:49 AM UTC, comment #1:

This is really part of a larger issue, in that wget doesn't handle IRIs in general. In order to do this properly, it'll need to do things like, remember what encoding the HTML file was served as (including when wget is invoked with -c at some point after the HTML file was parsed).

Micah Cowan <micahcowan>In charge of this item.
Mon 09 Jul 2007 09:52:21 PM UTC, original submission:

Hello,

wget doesn't support IDN domains:

$ LC_ALL=C wget 'http://www.köln.de/'
--23:49:06-- http://www.k%C3%B6ln.de/
=> `index.html'
Resolving http://www.k\303\266ln.de... failed: Name or service not known.

Translating the URL to the xn-- it works.

Maybe its possible to use GNU libidn http://www.gnu.org/software/libidn/

thx.

-unavailable-

Noël Köthe <nok>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by sxav (Updated the item)
  • -unavailable- added by micahcowan (Updated the item)
  • -unavailable- added by nok (Submitted the item)
  •  

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 14 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Thu 02 Jul 2009 05:46:03 AM UTCmicahcowanStatusReady For Test=>Fixed
      Open/ClosedOpen=>Closed
      Fixed ReleaseNone=>1.12
    Fri 22 Aug 2008 12:01:53 AM UTCmicahcowanPriority6=>9 - Immediate
      Assigned tosxav=>micahcowan
    Fri 15 Aug 2008 06:35:46 PM UTCsxavStatusConfirmed=>Ready For Test
    Mon 14 Apr 2008 04:04:30 AM UTCmicahcowanAssigned toNone=>sxav
    Tue 22 Jan 2008 11:42:06 PM UTCmicahcowanPriority5 - Normal=>6
    Tue 22 Jan 2008 11:21:39 PM UTCmicahcowanPlanned Release2.0=>1.12
    Wed 22 Aug 2007 04:25:49 AM UTCmicahcowanSummarywget doesn\'t handle IDN domains=>Wget should handle IRIs.
    Fri 20 Jul 2007 06:43:49 AM UTCmicahcowanCategoryFeature Request=>Protocol Issue
      Planned ReleaseNone=>2.0
    Mon 09 Jul 2007 10:13:17 PM UTCmicahcowanStatusNone=>Confirmed
      Work RequiredNone=>2 - Weeks

    Back to the top


    Powered by Savane 3.1-cleanup1