buglibiconv - Bugs: bug #42847, Incorrect conversion to/from...

 
 

bug #42847: Incorrect conversion to/from wchar_t strings on Win32

Submitted by:  Evgeny Grin <karlson2k>
Submitted on:  Sat 26 Jul 2014 12:31:26 PM UTC  
 
Category: EncodingsSeverity: 3 - Normal
Item Group: NoneStatus: Works For Me
Privacy: PublicAssigned to: Bruno Haible <haible>
Open/Closed: Closed

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

Sat 01 Oct 2016 05:03:09 PM UTC, comment #2:

The difference between utf16{le,be}.h and a hypothetical utf16internal.h would be that the latter assumes correct alignment. But:

The speed difference between 2 consecutive byte accesses and a 2-byte-word accesses should be really really tiny in modern CPUs.

Therefore I find this is not worth the risk of trouble:
- Do we know that all wchar_t in Windows are properly aligned?
- On x86 CPUs, unaligned accesses can be expensive. On other CPUs to which Windows has been ported (e.g. ARM), unaligned accesses are even forbidden.

Bruno Haible <haible>
Project AdministratorIn charge of this item.
Sat 26 Jul 2014 12:42:30 PM UTC, comment #1:

Sorry, this was fixed in 1.14, but sill is suboptimal.
Implementation of utf16internal.h will be better.

Evgeny Grin <karlson2k>
Sat 26 Jul 2014 12:31:26 PM UTC, original submission:

Win32 use UTF-16 for wchar_t (i.e. use surrogates pairs)
Libiconv use ucs2internal for win32, which results in rejecting all surrogates. Can be fast fixed by using utf16le or better fixed by implementing utf16internal.h.

Evgeny Grin <karlson2k>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by haible (Posted a comment)
  • -unavailable- added by karlson2k (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 4 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Wed 23 Nov 2016 11:54:02 PM UTChaibleCategoryNone=>Encodings
    Sat 01 Oct 2016 05:03:09 PM UTChaibleStatusNone=>Works For Me
      Assigned toNone=>haible
      Open/ClosedOpen=>Closed

    Back to the top


    Powered by Savane 3.1-cleanup1