bugGNU gettext - Bugs: bug #41668, xgettext incorrect parsing of...

 
 

You are not allowed to post comments on this tracker with your current authentication level.

bug #41668: xgettext incorrect parsing of python brace format

Submitter:  Kovid Goyal <kovidgoyal>
Submitted:  Sat 22 Feb 2014 01:14:46 PM UTC
   
 
Category:  Python Severity:  3 - Normal
Item Group:  None Status:  Fix Released
Privacy:  Public Assigned to:  ueno
Open/Closed:  Closed

Jump to the original submission

Wed 30 Apr 2014 04:34:07 AM UTC, comment #7: 
Daiki Ueno <ueno>
Group administrator
Wed 05 Mar 2014 09:24:28 AM UTC, comment #6: 

Well,

> msgfmt (even without -c) fails because of this.


are you sure about this?  Then, it sounds like another bug...
I can't reproduce it with the following input:


msgid ""
msgstr ""
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=ISO-8859-1\n"
"Content-Transfer-Encoding: 8-bit\n"

#, python-brace-format
msgid "cheese{0:<c>}"
msgstr "cheese"


Daiki Ueno <ueno>
Group administrator
Tue 04 Mar 2014 05:02:24 PM UTC, comment #5: 

To clarify the "poedit refuses to save a .po file" comment: that's not the case, but it fails to compile into .mo, because msgfmt (even without -c) fails because of this.

The only workaround I was able to find was to use an explicit # no-python-brace-format comment, which is usually good enough, but in this particular case, the .py file was generated, making this impossible.

Vaclav Slavik <vslavik>
Mon 24 Feb 2014 09:13:34 AM UTC, comment #4: 

Thanks for the background information.  Then I think the imminent problem could be avoided by recognizing "Standard Format Specifiers" in PEP3101 more precisely, to reject format specifiers like "</c>".  Let's fix it before the next release.

Daiki Ueno <ueno>
Group administrator
Mon 24 Feb 2014 03:22:34 AM UTC, comment #3: 

It doesn't cause me any hardship personally, since I just regexed the special case in the code that calls xgettext as part of my build process. The reason for the bug report is that poedit refuses to save a .po file generated from this .pot file.

Since marking a string incorrectly as containing brace formatting appears to break at least one translation tool, I though you should know about it.

The general solution to this problem would be to lift the python code that parses strings for brace formatting and use it (I'm fairly certain it would be implemented in C for performance). That way you would only mark strings as having brace formatting if python itself thought they had valid brace formatting. Which would mitigate the incidence of falsely marked strings. But that may be too much work.


Below is the quote from the poedit developer:

------------------------------------------------------------

Hi Paulo,

thanks,  that helped. Unfortunately, it is a problem with the file from
calibre — it incorrectly marks this string as "python-brace-format”, which
means that portions of it enclosed inside {…} are treated specially and
the translation doesn’t fit what GNU gettext expects here.

I attached manually corrected PO file where I replaced it with
“no-python-brace-format”, but the problem will reappear if you ever use an
updated PO(T) file from calibre. The “real” fix, once for all, is to fix
it in calibre’s source code, in create_custom_column_ui.py. It may be a
good idea to mention this when sending the translation to them.

Kovid Goyal <kovidgoyal>
Mon 24 Feb 2014 03:13:07 AM UTC, comment #2: 

I doubt if we could do much on this.   To parse your first example correctly, we will need to recognize .format() call.  That is not easy to do with the current scanner.  On the second example, it's out of scope to support invalid python expressions.

Anyway, do you have any real problem (e.g. msgfmt signals error)?  If so, perhaps we could think of some workaround...

Daiki Ueno <ueno>
Group administrator
Sat 22 Feb 2014 01:20:19 PM UTC, comment #1: 

Here is minimal example:

_("{0:<c>}").format(1)

raises the following exception in python:
ValueError: Invalid conversion specification

Demonstrating that it is not a valid brace format specifier, xgettext nonetheless marks it as python-brace-format

Kovid Goyal <kovidgoyal>
Sat 22 Feb 2014 01:14:46 PM UTC, original submission:  

xgettext incorrect marks the following string as using python-brace-format:

_("<p>The format specifier must begin with <code>{0:</code>\n"
"and end with <code>}</code> You can have text before and after the format specifier.\n")

Kovid Goyal <kovidgoyal>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by haible (Updated the item)
  • -email is unavailable- added by ueno (Posted a comment)
  • -email is unavailable- added by kovidgoyal (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

     

    Follow 6 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2023-01-04 haible StatusFixed Fix Released
    2016-11-27 haible CategoryNone Python
    2014-04-30 ueno StatusConfirmed Fixed
        Open/ClosedOpen Closed
    2014-02-24 ueno StatusNone Confirmed
        Assigned toNone ueno

    Back to the top

    Powered by Savane 3.13-3230.
    Corresponding source code