bugGNU gettext - Bugs: bug #41668, xgettext incorrect parsing of...

 
 

You are not allowed to post comments on this tracker with your current authentication level.

bug #41668: xgettext incorrect parsing of python brace format

Submitted by:  Kovid Goyal <kovidgoyal>
Submitted on:  Sat 22 Feb 2014 01:14:46 PM UTC  
 
Category: PythonSeverity: 3 - Normal
Item Group: NoneStatus: Fixed
Privacy: PublicAssigned to: Daiki Ueno <ueno>
Open/Closed: Closed

(Jump to the original submission Jump to the original submission)

Wed 30 Apr 2014 04:34:07 AM UTC, comment #7:

So I ended up with this:

http://git.savannah.gnu.org/cgit/gettext.git/commit/?h=maint&id=4dc36d2f6a4a544cce6cd0e9dc45a080139cd9d1

Daiki Ueno <ueno>
Project AdministratorIn charge of this item.
Wed 05 Mar 2014 09:24:28 AM UTC, comment #6:

Well,

> msgfmt (even without -c) fails because of this.


are you sure about this? Then, it sounds like another bug...
I can't reproduce it with the following input:

Daiki Ueno <ueno>
Project AdministratorIn charge of this item.
Tue 04 Mar 2014 05:02:24 PM UTC, comment #5:

To clarify the "poedit refuses to save a .po file" comment: that's not the case, but it fails to compile into .mo, because msgfmt (even without -c) fails because of this.

The only workaround I was able to find was to use an explicit # no-python-brace-format comment, which is usually good enough, but in this particular case, the .py file was generated, making this impossible.

Vaclav Slavik <vslavik>
Mon 24 Feb 2014 09:13:34 AM UTC, comment #4:

Thanks for the background information. Then I think the imminent problem could be avoided by recognizing "Standard Format Specifiers" in PEP3101 more precisely, to reject format specifiers like "</c>". Let's fix it before the next release.

Daiki Ueno <ueno>
Project AdministratorIn charge of this item.
Mon 24 Feb 2014 03:22:34 AM UTC, comment #3:

It doesn't cause me any hardship personally, since I just regexed the special case in the code that calls xgettext as part of my build process. The reason for the bug report is that poedit refuses to save a .po file generated from this .pot file.

Since marking a string incorrectly as containing brace formatting appears to break at least one translation tool, I though you should know about it.

The general solution to this problem would be to lift the python code that parses strings for brace formatting and use it (I'm fairly certain it would be implemented in C for performance). That way you would only mark strings as having brace formatting if python itself thought they had valid brace formatting. Which would mitigate the incidence of falsely marked strings. But that may be too much work.

Below is the quote from the poedit developer:

------------------------------------------------------------

Hi Paulo,

thanks, that helped. Unfortunately, it is a problem with the file from
calibre — it incorrectly marks this string as "python-brace-format”, which
means that portions of it enclosed inside {…} are treated specially and
the translation doesn’t fit what GNU gettext expects here.

I attached manually corrected PO file where I replaced it with
“no-python-brace-format”, but the problem will reappear if you ever use an
updated PO(T) file from calibre. The “real” fix, once for all, is to fix
it in calibre’s source code, in create_custom_column_ui.py. It may be a
good idea to mention this when sending the translation to them.

Kovid Goyal <kovidgoyal>
Mon 24 Feb 2014 03:13:07 AM UTC, comment #2:

I doubt if we could do much on this. To parse your first example correctly, we will need to recognize .format() call. That is not easy to do with the current scanner. On the second example, it's out of scope to support invalid python expressions.

Anyway, do you have any real problem (e.g. msgfmt signals error)? If so, perhaps we could think of some workaround...

Daiki Ueno <ueno>
Project AdministratorIn charge of this item.
Sat 22 Feb 2014 01:20:19 PM UTC, comment #1:

Here is minimal example:

_("{0:<c>}").format(1)

raises the following exception in python:
ValueError: Invalid conversion specification

Demonstrating that it is not a valid brace format specifier, xgettext nonetheless marks it as python-brace-format

Kovid Goyal <kovidgoyal>
Sat 22 Feb 2014 01:14:46 PM UTC, original submission:

xgettext incorrect marks the following string as using python-brace-format:

_("<p>The format specifier must begin with <code>{0:</code>\n"
"and end with <code>}</code> You can have text before and after the format specifier.\n")

Kovid Goyal <kovidgoyal>

 

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by haible (Updated the item)
  • -unavailable- added by ueno (Posted a comment)
  • -unavailable- added by kovidgoyal (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 5 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Sun 27 Nov 2016 11:43:56 AM UTChaibleCategoryNone=>Python
    Wed 30 Apr 2014 04:34:07 AM UTCuenoStatusConfirmed=>Fixed
      Open/ClosedOpen=>Closed
    Mon 24 Feb 2014 09:13:34 AM UTCuenoStatusNone=>Confirmed
      Assigned toNone=>ueno

    Back to the top


    Powered by Savane 3.1-cleanup1