bugGNU gettext - Bugs: bug #24025, msgfmt --check-format allows...

 
 

You are not allowed to post comments on this tracker with your current authentication level.

bug #24025: msgfmt --check-format allows omitted format descriptors in plural forms for Python

Submitted by:  Alexander Dupuy <dupuy>
Submitted on:  Fri 08 Aug 2008 12:13:07 PM UTC  
 
Category: Plural formsSeverity: 3 - Normal
Item Group: NoneStatus: Fixed
Privacy: PublicAssigned to: Bruno Haible <haible>
Open/Closed: Closed

Fri 15 Aug 2008 10:29:58 AM UTC, comment #1:

> It is quite common in Arabic (and possibly other languages) that an idiomatic
> translation of plural forms requires omitting the actual numeric digits for
> one or more of the plural forms (in Arabic, these are typically the forms for
> values of 1 or 2).


Yes, this is known.

> In applications written in C, it is possible to simply omit the %d in those
> translations, and printf happily ignores the presence of the additional
> arguments. (This approach doesn't work when there are other format specifiers
> after the %d in the string, e.g. "There are %d files in directory '%s'").
>
> To support this usage, msgfmt --check-format allows specific plural form
> translations to omit a format specifier. (There is actually rather clever
> code to determine if a plural form is used for many values, and in these
> cases, a strict equality check ensures that omissions are flagged.)


Correct.

> In Python, however, the string formatting % operator raises an exception if
> there are unused (unnamed) arguments:
>
> TypeError: not all arguments converted during string formatting
>
> The GNU gettext tools format parser for Python allows for fewer unnamed
> format strings in the case of specific plural forms, but it really should flag
> this as an error.


Agreed. I wasn't aware of this Python error message. Fixed in the gettext CVS.

xgettext already emits a warning that for plural form handling, named
arguments syntax is preferred over unnamed arguments.

>>> print "%(count)x pieces of %(brand)s cake" % { 'count': 17, 'brand': "muffin" }

11 pieces of muffin cake

>>> print "A%(count).0s piece of %(brand)s cake" % { 'count': 17, 'brand': "muffin" }

A piece of muffin cake

>>> print "A piece of %(brand)s cake" % { 'count': 17, 'brand': "muffin" }

A piece of muffin cake

I'm also adding this advice to the gettext documentation.

> Although Python does not allow you to omit the format specifier, there is a
> way to put a placeholder format descriptor (%.0s) that expands to an empty
> string, giving much the same effect. Since Python will implicitly perform
> string conversion on the numeric argument, this placeholder format descriptor
> is perfectly safe to use (it also works with some C printf implementations,
> but might segfault with others).
>
> However, msgfmt --check-format will flag a use of this placeholder as a fatal
> error, because the original %d or %i has been replaced with %.0s, which has a
> different type.


I wasn't aware of this either. I'm changing format-python.c to support this
.0s and .0r special case.

> To allow this placeholder use, format-python.c:format_check() could be
> modified so that if "strict" (equality) is not true, a (spec2.unnamed[i].type
> == FAT_PLACEHOLDER) would not cause a mismatch. FAT_PLACEHOLDER would be a
> new type that would be assigned by the format_parse() function for the format
> string %.0s.


Right. I'm using FAT_ANY for this purpose.

Bruno Haible <haible>
Project AdministratorIn charge of this item.
Fri 08 Aug 2008 12:13:07 PM UTC, original submission:

It is quite common in Arabic (and possibly other languages) that an idiomatic translation of plural forms requires omitting the actual numeric digits for one or more of the plural forms (in Arabic, these are typically the forms for values of 1 or 2).

In applications written in C, it is possible to simply omit the %d in those translations, and printf happily ignores the presence of the additional arguments. (This approach doesn't work when there are other format specifiers after the %d in the string, e.g. "There are %d files in directory '%s'").

To support this usage, msgfmt --check-format allows specific plural form translations to omit a format specifier. (There is actually rather clever code to determine if a plural form is used for many values, and in these cases, a strict equality check ensures that omissions are flagged.)

In Python, however, the string formatting % operator raises an exception if there are unused (unnamed) arguments:

TypeError: not all arguments converted during string formatting

The GNU gettext tools format parser for Python allows for fewer unnamed format strings in the case of specific plural forms, but it really should flag this as an error.

If gettext-0.17/gettext-tools/src/format-python.c:format_check() lines 496-513 were revised so that regardless of the "strict" (equality) parameter, (spec1->unnamed_arg_count != spec2->unnamed_arg_count) causes an error, this potential error would be flagged by msgfmt --check-format.

Although Python does not allow you to omit the format specifier, there is a way to put a placeholder format descriptor (%.0s) that expands to an empty string, giving much the same effect. Since Python will implicitly perform string conversion on the numeric argument, this placeholder format descriptor is perfectly safe to use (it also works with some C printf implementations, but might segfault with others).

However, msgfmt --check-format will flag a use of this placeholder as a fatal error, because the original %d or %i has been replaced with %.0s, which has a different type.

To allow this placeholder use, format-python.c:format_check() could be modified so that if "strict" (equality) is not true, a (spec2.unnamed[i].type == FAT_PLACEHOLDER) would not cause a mismatch. FAT_PLACEHOLDER would be a new type that would be assigned by the format_parse() function for the format string %.0s.

I would be able to submit a patch that implements these changes, but wanted to know if these changes would be accepted.

@alex

Alexander Dupuy <dupuy>

 

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by haible (Posted a comment)
  • -unavailable- added by dupuy (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 4 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Sat 26 Nov 2016 11:59:34 PM UTChaibleCategoryNone=>Plural forms
    Fri 15 Aug 2008 10:29:58 AM UTChaibleStatusNone=>Fixed
      Assigned toNone=>haible
      Open/ClosedOpen=>Closed

    Back to the top


    Powered by Savane 3.1-cleanup1