bugGNU gettext - Bugs: bug #57990, msgmerge (not msgfmt) invalid...


You are not allowed to post comments on this tracker with your current authentication level.

bug #57990: msgmerge (not msgfmt) invalid multibyte sequence for valid UTF-8 sequence

Submitted by:  Sam Liddicott <samjam>
Submitted on:  Fri 13 Mar 2020 03:51:53 PM UTC  
Category:  Translator tools Severity:  3 - Normal
Item Group:  None Status:  Not a Bug
Privacy:  Public Assigned to:  Bruno Haible <haible>
Open/Closed:  Closed

Mon 16 Mar 2020 05:59:49 PM UTC, comment #3: 

I tend to think that it's well enough documented:

https://www.gnu.org/software/gettext/manual/html_node/PO-Files.html talks about the header entry and references
for the details.

Bruno Haible <haible>
Project AdministratorIn charge of this item.
Mon 16 Mar 2020 03:58:02 PM UTC, comment #2: 

Thanks Bruno, you definitely put your finger on the problem!

Perhaps we should consider this a documentation bug?

I don't find that the man pages mention anything useful about this, and the info docs for msgmerge mention it lightly in terms of meta information and header fields but with the general assumption that someone will have created a .pot file with the right headers

The info entry for "gettext" mentions it in more detail.

I merely suggest that more prominence could be given in the msgmerge man page, or where the error message is raised.

But I am more than content with your response, thankyou

Sam Liddicott <samjam>
Fri 13 Mar 2020 05:58:51 PM UTC, comment #1: 

The reason is that your PO file has no header entry; without a header entry the character encoding is not known and is therefore assumed to be ASCII.

When I add a minimal header entry (as in the attached file in.po), "msgmerge in.po /dev/null" succeeds.

> My environment has: LANG=en_GB.UTF-8

It's good that you state it. But for the translator tools, the LANG etc. environment variables don't influence the msgmerge result.

(file #48592)

Bruno Haible <haible>
Project AdministratorIn charge of this item.
Fri 13 Mar 2020 03:51:53 PM UTC, original submission:  

Using msgmerge this simple test fails:

echo bXNnaWQgIldBUk5JTkciCm1zZ3N0ciAi6K2m5ZGKIgo= | base64 -d | msgmerge /dev/stdin /dev/null

even using iconv to sanitise fails, though iconv doesn't complain:

echo bXNnaWQgIldBUk5JTkciCm1zZ3N0ciAi6K2m5ZGKIgo= | base64 -d | iconv -t UTF-8 | msgmerge /dev/stdin /dev/null

but the same input to msgfmt works:

echo bXNnaWQgIldBUk5JTkciCm1zZ3N0ciAi6K2m5ZGKIgo= | base64 -d | msgfmt -o /tmp/x.mo /dev/stdin

My system is LinuxMint 19.1

My environment has:

Sam Liddicott <samjam>


Attached Files
file #48592:  in.po added by haible (118B - text/x-gettext-translation)
file #48591:  x.po added by samjam (32B - text/x-gettext-translation - x.po is the failing file, but passed as valid by iconv and online utf-8 validators)


Depends on the following items: None found

Items that depend on this one: None found


Carbon-Copy List
  • -email is unavailable- added by haible (Updated the item)
  • -email is unavailable- added by samjam (Submitted the item)

    Do you think this task is very important?
    If so, you can add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.




    Follow 5 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2020-03-13 haible Attached File- => Added in.po, #48592
        StatusNone => Not a Bug
        Assigned toNone => haible
        Open/ClosedOpen => Closed
    2020-03-13 samjam Attached File- => Added x.po, #48591

    Back to the top

    Powered by Savane 3.5