bugGNU gettext - Bugs: bug #48481, [PATCH] Allow CLDR plural rules...

 
 

You are not allowed to post comments on this tracker with your current authentication level.

bug #48481: [PATCH] Allow CLDR plural rules lookup fallback

Submitted by:  Michele Locati <mlocati>
Submitted on:  Tue 12 Jul 2016 08:14:53 AM UTC  
 
Category: Plural formsSeverity: 3 - Normal
Item Group: NoneStatus: None
Privacy: PublicAssigned to: None
Open/Closed: Open

(Jump to the original submission Jump to the original submission)

Thu 04 Aug 2016 09:11:02 AM UTC, comment #7:

What about something like this http://cpp.sh/9hov3f ?

PS: I moved the code to https://github.com/mlocati/gettext-unicode-locale-id

Michele Locati <mlocati>
Wed 03 Aug 2016 08:10:20 PM UTC, comment #6:

Just adding some data points, you might want to look at the recent efforts to adopt CLDR script names in glibc:
https://sourceware.org/ml/libc-locales/2016-q2/msg00034.html
and some *BSD variants:
https://wiki.freebsd.org/LocaleNewApproach

Daiki Ueno <ueno>
Project Administrator
Wed 03 Aug 2016 04:29:10 PM UTC, comment #5:

I've had some time to take a look at this.

What about adding some helper functions to work with locale identifiers in Gettext and Unicode formats?

I mean, something like this: http://cpp.sh/7smfb (attached file: parse-locale-identifiers.c )

(file #38093)

Michele Locati <mlocati>
Wed 03 Aug 2016 01:46:00 PM UTC, comment #4:

Now I understand what you mean.

BTW, I don't understand all the components of the locale identifiers as defined by http://git.savannah.gnu.org/cgit/gettext.git/tree/gettext-runtime/intl/finddomain.c#n71

For Unicode we have this definition: http://unicode.org/reports/tr35/#Unicode_language_identifier

Here's an example of an Unicode ID with all the fields:
it-Latn-IT-POSIX-NYNORSK

Where:
it: language subtag
Latn: script subtag
IT: region subtag
POSIX, NYNORSK: variant subtags

For gettext IDs, we have:
language[_territory][.codeset][@modifier]
What are the values of codeset and modifier? Do their meaning overlaps somehow with script/variant as defined by Unicode?

Michele Locati <mlocati>
Wed 03 Aug 2016 08:18:47 AM UTC, comment #3:

Yes, that's what I understand from your code, and I wonder if it is a reasonable behaviour; shouldn't it signal an error or stop processing when it sees a '@' twice?

By the way, I think there should be a reliable conversion between CLDR locale names and gettext locale names.

Daiki Ueno <ueno>
Project Administrator
Wed 03 Aug 2016 07:57:56 AM UTC, comment #2:

My code build alternative IDs looking for these chars: '.', '_', '-', '@'.
So, if locale is "foo@bar@baz", we have this list of IDs:
- "foo@bar@baz"
- "foo@bar"
- "foo"

Instead of storing copies of the string, I save the position of the stop chars. In the above case, we'll have these values:
- locale_lengths_count = 3
- locale_lengths[0] = 11
- locale_lengths[1] = 7
- locale_lengths[2] = 3

When looking for the plural rule, the code compares the IDs using each of these locale_lengths using xmlStrncmp and stops at the first match.

So, if someone specify "en@us", the code won't convert it to "en-us", but simply looks for "en@us" and "en".

Michele Locati <mlocati>
Wed 03 Aug 2016 07:16:12 AM UTC, comment #1:

Thank you for the patch, it makes sense.

Regarding the implementation, though I haven't tested the patch, I wonder if it accepts bogus locales like: foo@bar@baz? Also doesn't it stop at the second '-' if the locale looks like: it-IT.UTF-8?

I would like to make it more strictly conforming to the existing logic:
http://git.savannah.gnu.org/cgit/gettext.git/tree/gettext-runtime/intl/finddomain.c#n71

Would you mind checking the above cases, and if possible add a similar comment there?

Daiki Ueno <ueno>
Project Administrator
Tue 12 Jul 2016 08:14:53 AM UTC, original submission:

Let's allow the CLDR plural rules lookup to fallback to "parent" locale ids.
For instance, looking for "it_IT.UTF-8" will also find "it_IT" and "it".

Michele Locati <mlocati>

 

Attached Files

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by haible (Updated the item)
  • -unavailable- added by ueno (Posted a comment)
  • -unavailable- added by mlocati (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 3 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Sun 27 Nov 2016 12:15:55 PM UTChaibleCategoryNone=>Plural forms
    Wed 03 Aug 2016 04:29:10 PM UTCmlocatiAttached File-=>Added parse-locale-identifiers.c, #38093
    Tue 12 Jul 2016 08:14:53 AM UTCmlocatiAttached File-=>Added 0001-Allow-CLDR-plural-rules-lookup-fallback.patch, #37823

    Back to the top


    Powered by Savane 3.1-cleanup1