bugGNU gettext - Bugs: bug #65104, msgmerge fails to merge utf8 PO...

 
 

You are not allowed to post comments on this tracker with your current authentication level.

bug #65104: msgmerge fails to merge utf8 PO file against an iso-8859-1 POT file

Submitter:  Martin Quinson <mquinson>
Submitted:  Tue 02 Jan 2024 01:07:45 AM UTC
   
 
Category:  Translator tools Severity:  3 - Normal
Item Group:  None Status:  Answer Provided
Privacy:  Public Assigned to:  haible
Open/Closed:  Closed

Sun 13 Oct 2024 05:00:07 PM UTC, comment #1: 

Thanks for the report.

The problem is that iso8859.pot is used as a POT file, contains non-ASCII characters, but is not in UTF-8 encoding.

Since, by https://www.gnu.org/software/gettext/manual/html_node/Creating.html, a translator is supposed to be able to copy the POT file to a PO file and use that as a starting point for her translation, the file ought to be in UTF-8 encoding. If it is in ISO-8859-1 encoding, how would e.g. a Bulgarian or Chinese translator do this? Most likely she would produce mojibake, then convert the POT file to UTF-8 and start over. Translators should not encounter such hassles.

I am therefore adding an error message for this case, in https://git.savannah.gnu.org/gitweb/?p=gettext.git;a=commitdiff;h=bd94293cc8ee0ba818cff4209ab4c4466c25189a .

Your example will now produce this error message:

$ LC_ALL=C msgmerge utf8.po iso8859.pot
msgmerge: iso8859.pot: The file contains non-ASCII characters but the present charset "ISO-8859-1" is not UTF-8.


Bruno Haible <haible>
Group administrator
Tue 02 Jan 2024 01:07:45 AM UTC, original submission:  

Hello,

when I try to merge a file encoded in utf8 against a pot file that is encoded in latin1, I get an error message about invalid multibyte strings, and the non-ascii chars of the msgids are mangled.

Here are two files to reproduce the problem.

$ file *
iso8859.pot: GNU gettext message catalogue, ISO-8859 text
utf8.po:     GNU gettext message catalogue, Unicode text, UTF-8 text



$ grep charset= *
iso8859.pot:"Content-Type: text/plain; charset=ISO-8859-1\n"
utf8.po:"Content-Type: text/plain; charset=UTF-8\n"



$ iconv -f iso-8859-1 -t UTF-8 iso8859.pot -o /dev/null
$ iconv -f UTF-8 -t latin1 utf8.po -o /dev/null



$ LC_ALL=C msgmerge utf8.po iso8859.pot
..... done.
# Language up translations for po package
# Copyright (C) 2020 Free Software Foundation, Inc.
# This file is distributed under the same license as the po package.
# Automatically generated, 2020.
#
msgid ""
msgstr ""
"Project-Id-Version: po 4a\n"
"POT-Creation-Date: 2024-01-02 02:22+0100\n"
"PO-Revision-Date: 2020-04-09 17:33+0200\n"
"Last-Translator: Automatically generated\n"
"Language-Team: none\n"
"Language: up\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"

#. type: =head1
#: iso8859.pod:1
#, fuzzy
utf8.po:21: invalid multibyte sequence
utf8.po:21: invalid multibyte sequence
msgid "Ttulo de prueba"
msgstr "TÍTULO DE PRUEBA"

#. type: textblock
#: iso8859.pod:3
#, fuzzy
utf8.po:26: invalid multibyte sequence
utf8.po:26: invalid multibyte sequence
utf8.po:26: invalid multibyte sequence
utf8.po:26: invalid multibyte sequence
msgid "blbleble llalala"
msgstr "BLÈBLEBLE LÁLALALA"

Martin Quinson <mquinson>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attached Files
file #55503:  utf8.po added by mquinson (735B - text/x-gettext-translation)
file #55504:  iso8859.pot added by mquinson (700B - application/vnd.ms-powerpoint)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by haible (Updated the item)
  • -email is unavailable- added by mquinson (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

     

    Follow 6 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2024-10-13 haible StatusNone Answer Provided
        Assigned toNone haible
        Open/ClosedOpen Closed
    2024-07-25 haible CategoryNone Translator tools
    2024-01-02 mquinson Attached File- Added utf8.po, #55503
        Attached File- Added iso8859.pot, #55504

    Back to the top

    Powered by Savane 3.15-e6e5.
    Corresponding source code