bugGNU gettext - Bugs: bug #57990, msgmerge (not msgfmt) invalid...


You are not allowed to post comments on this tracker with your current authentication level.

bug #57990: msgmerge (not msgfmt) invalid multibyte sequence for valid UTF-8 sequence

Submitter:  Sam Liddicott <samjam>
Submitted:  Fri 13 Mar 2020 03:51:53 PM UTC
Category:  Translator tools Severity:  3 - Normal
Item Group:  None Status:  Not a Bug
Privacy:  Public Assigned to:  haible
Open/Closed:  Closed
* Mandatory Fields

Add a New Comment (Rich Markup)

Comment Type & Canned Response:
       No canned response available


Mon 16 Mar 2020 05:59:49 PM UTC, comment #3: 

I tend to think that it's well enough documented:

https://www.gnu.org/software/gettext/manual/html_node/PO-Files.html talks about the header entry and references
for the details.

Bruno Haible <haible>
Project Administrator
Mon 16 Mar 2020 03:58:02 PM UTC, comment #2: 

Thanks Bruno, you definitely put your finger on the problem!

Perhaps we should consider this a documentation bug?

I don't find that the man pages mention anything useful about this, and the info docs for msgmerge mention it lightly in terms of meta information and header fields but with the general assumption that someone will have created a .pot file with the right headers

The info entry for "gettext" mentions it in more detail.

I merely suggest that more prominence could be given in the msgmerge man page, or where the error message is raised.

But I am more than content with your response, thankyou

Sam Liddicott <samjam>
Fri 13 Mar 2020 05:58:51 PM UTC, comment #1: 

The reason is that your PO file has no header entry; without a header entry the character encoding is not known and is therefore assumed to be ASCII.

When I add a minimal header entry (as in the attached file in.po), "msgmerge in.po /dev/null" succeeds.

> My environment has: LANG=en_GB.UTF-8

It's good that you state it. But for the translator tools, the LANG etc. environment variables don't influence the msgmerge result.

(file #48592)

Bruno Haible <haible>
Project Administrator
Fri 13 Mar 2020 03:51:53 PM UTC, original submission:  

Using msgmerge this simple test fails:

echo bXNnaWQgIldBUk5JTkciCm1zZ3N0ciAi6K2m5ZGKIgo= | base64 -d | msgmerge /dev/stdin /dev/null

even using iconv to sanitise fails, though iconv doesn't complain:

echo bXNnaWQgIldBUk5JTkciCm1zZ3N0ciAi6K2m5ZGKIgo= | base64 -d | iconv -t UTF-8 | msgmerge /dev/stdin /dev/null

but the same input to msgfmt works:

echo bXNnaWQgIldBUk5JTkciCm1zZ3N0ciAi6K2m5ZGKIgo= | base64 -d | msgfmt -o /tmp/x.mo /dev/stdin

My system is LinuxMint 19.1

My environment has:

Sam Liddicott <samjam>


(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:

Attached Files
file #48592:  in.po added by haible (118B - text/x-gettext-translation)
file #48591:  x.po added by samjam (32B - text/x-gettext-translation - x.po is the failing file, but passed as valid by iconv and online utf-8 validators)


Depends on the following items: None found

Items that depend on this one: None found


Carbon-Copy List
  • -email is unavailable- added by haible (Updated the item)
  • -email is unavailable- added by samjam (Submitted the item)

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.





    Follow 5 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2020-03-13 haible Attached File- Added in.po, #48592
        StatusNone Not a Bug
        Assigned toNone haible
        Open/ClosedOpen Closed
    2020-03-13 samjam Attached File- Added x.po, #48591

    Back to the top

    Powered by Savane 3.9