buglibiconv - Bugs: bug #42847, Incorrect conversion to/from...

 
 

bug #42847: Incorrect conversion to/from wchar_t strings on Win32

Submitter:  Evgeny Grin <karlson2k>
Submitted:  Sat 26 Jul 2014 12:31:26 PM UTC
   
 
Category:  Encodings Severity:  3 - Normal
Item Group:  None Status:  Works For Me
Privacy:  Public Assigned to:  haible
Open/Closed:  Closed
* Mandatory Fields

Add a New Comment Rich Markup
   

Sat 01 Oct 2016 05:03:09 PM UTC, comment #2: 

The difference between utf16{le,be}.h and a hypothetical utf16internal.h would be that the latter assumes correct alignment. But:

The speed difference between 2 consecutive byte accesses and a 2-byte-word accesses should be really really tiny in modern CPUs.

Therefore I find this is not worth the risk of trouble:
- Do we know that all wchar_t in Windows are properly aligned?
- On x86 CPUs, unaligned accesses can be expensive. On other CPUs to which Windows has been ported (e.g. ARM), unaligned accesses are even forbidden.

Bruno Haible <haible>
Group administrator
Sat 26 Jul 2014 12:42:30 PM UTC, comment #1: 

Sorry, this was fixed in 1.14, but sill is suboptimal.
Implementation of utf16internal.h will be better.

Evgeny Grin <karlson2k>
Sat 26 Jul 2014 12:31:26 PM UTC, original submission:  

Win32 use UTF-16 for wchar_t (i.e. use surrogates pairs)
Libiconv use ucs2internal for win32, which results in rejecting all surrogates. Can be fast fixed by using utf16le or better fixed by implementing utf16internal.h.

Evgeny Grin <karlson2k>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by haible (Posted a comment)
  • -email is unavailable- added by karlson2k (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 4 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2016-11-23 haible CategoryNone Encodings
    2016-10-01 haible StatusNone Works For Me
        Assigned toNone haible
        Open/ClosedOpen Closed

    Back to the top

    Powered by Savane 3.13-f8d8.
    Corresponding source code