bugGNU gettext - Bugs: bug #57714, libintl: Unicode paths passed to...

 
 

You are not allowed to post comments on this tracker with your current authentication level.

bug #57714: libintl: Unicode paths passed to bindtextdomain fail on Windows (when linked to the MSVCRT)

Submitter:  pastdue
Submitted:  Fri 31 Jan 2020 07:51:03 PM UTC
   
 
Category:  End-user / runtime Severity:  3 - Normal
Item Group:  None Status:  Fix Released
Privacy:  Public Assigned to:  haible
Open/Closed:  Closed

Mon 27 Jul 2020 05:44:59 PM UTC, comment #2: 

The fix is contained in gettext 0.21.

Bruno Haible <haible>
Group administrator
Mon 20 Apr 2020 02:04:02 AM UTC, comment #1: 

Thanks for the report.

Fixed through https://git.savannah.gnu.org/gitweb/?p=gettext.git;a=commitdiff;h=2e397fbb43fa0c6008897b50ccd2b9ffcd0a0cb5

I chose a different approach than yours, because

  • On native Windows, 'char *' strings are supposed to be in the encoding returned by GetACP(), not in UTF-8.
  • Therefore changing the way bindtextdomain() works would not only be backward-incompatible, but would also be inconsistent with Windows coding conventions.
  • Windows coding conventions imply that file names and directory names should be represented by wchar_t arrays, if they may contain arbitrary Unicode characters.


Bruno Haible <haible>
Group administrator
Fri 31 Jan 2020 07:51:03 PM UTC, original submission:  

Description:

libintl fails to load message catalogs from dirnames containing non-ASCII UTF-8 characters

Environment: Windows
Compiler: MSVC, & presumably anything else that links to (any version of) the MSVCRT

To Reproduce:

With a Windows build linked to the MSVCRT:

  • Call bindtextdomain with a UTF-8 dirname, containing characters outside of the ASCII range (as works on other platforms)


Expected Behavior:

libintl succeeds at finding message catalogs in the valid UTF-8 dirname

Actual Behavior:

libintl fails to load message catalogs

Cause:

gettext-runtime/intl/loadmsgcat.c uses open() internally, which does not support / expect UTF-8 filenames on the MSVCRT

Solution:

gettext-runtime/intl/loadmsgcat.c: Provide a wrapper for open() that converts the input char* to a UTF-16 wchar_t* and then calls _wopen(), which properly handles a Unicode path

I've attached an example patch (for an earlier version of libintl).
But note: the example patch requires linking to kernel32.lib on Windows for MultiByteToWideChar. If you have an alternative recommendation for converting UTF-8 char* to UTF-16 wchar_t* that would fit an upstream patch better, please let me know.

pastdue

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attached Files
file #48317:  example_patch.txt added by pastdue (1KiB - text/plain)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by haible (Posted a comment)
  • -email is unavailable- added by pastdue (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

     

    Follow 5 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2023-01-04 haible StatusFixed Fix Released
    2020-04-20 haible StatusNone Fixed
        Assigned toNone haible
        Open/ClosedOpen Closed
    2020-01-31 pastdue Attached File- Added example_patch.txt, #48317

    Back to the top

    Powered by Savane 3.13-d3ae.
    Corresponding source code