bugGNU gettext - Bugs: bug #53071, JSON as alternative to (G)MO

 
 

You are not allowed to post comments on this tracker with your current authentication level.

bug #53071: JSON as alternative to (G)MO

Submitter:  Guido Flohr <gflohr>
Submitted:  Mon 05 Feb 2018 04:11:45 PM UTC
   
 
Category:  JavaScript Severity:  1 - Wish
Item Group:  None Status:  None
Privacy:  Public Assigned to:  None
Open/Closed:  Open

Tue 06 Feb 2018 09:58:21 PM UTC, comment #5: 


> called it mo2json


Yes, you started a discussion about it in sr #109175 .

Bruno Haible <haible>
Group administrator
Tue 06 Feb 2018 08:57:12 PM UTC, comment #4: 

I did rewrote msgunfmt a year ago to achieve this together with JED.js and called it mo2json. Lemme check if I can find a patch tomorrow.

Markus Gothe <nihilus>
Tue 06 Feb 2018 12:41:56 AM UTC, comment #3: 


> Why? (g)mo is just a container format. Why should json for translation catalogs be more?


Good point. Maybe you are right.

> > Still, we should deal with LINE SEPARATOR and PARAGRAPH SEPARATOR


I'm referring to http://timelessrepo.com/json-isnt-a-javascript-subset and https://www.ietf.org/mail-archive/web/json/current/msg03146.html .

Bruno Haible <haible>
Group administrator
Mon 05 Feb 2018 10:41:18 PM UTC, comment #2: 


> Plural handling may need a bit more thought: where does the plural formula come from? How about:

{"disk":{plural:["Platte","Platten"], formula:"n==1?0:1"}}

Why? (g)mo is just a container format. Why should json for translation catalogs be more?

I think, there is a parser implementation for the "Plural-Forms" header for all programming languages that support a gettext runtime with ngettext().  For JavaScript, I have hacked together https://github.com/gflohr/locale-textdomain/blob/master/src/plural-exp.js some time ago, which should experience more testing, but basically works as far as I remember.

I personally wouldn't tie a possible JSON output format too tightly to JavaScript.  JavaScript is just the origin of JSON.  Why not leave the interpretation of a plural form formula to the consumer of the JSON?  In MO files it is also just a string literal.

> Parsing and validating a JSON of a specific schema can be done without an extra JSON library (see x-rst.c); it's only the parsing of generic JSON that would require a JSON library. That's what I inferred after looking at a couple of JSON libraries for C.


Great.  I thought it was more difficult.

> Still, we should deal with LINE SEPARATOR and PARAGRAPH SEPARATOR in such a way that the resulting JSON can be embedded in JavaScript (so that the client can use eval to parse it)


I didn't understand that, sorry.

Guido Flohr <gflohr>
Group Member
Mon 05 Feb 2018 09:51:32 PM UTC, comment #1: 

This is a very good proposal. Yes, I agree that JSON is the de-facto standard for server->client communication of data to JavaScript programs running in a browser.

> var parts = key.split("\004", 2);


Yes, contexts would be easy to adapt.

> {"Hello, world!":["Bonjour le monde!"]}


Plural handling may need a bit more thought: where does the plural formula come from? How about:
  {"disk":{plural:["Platte","Platten"], formula:"n==1?0:1"}}

> An new msgfmt option '--json'


Yes, and the same option also for msgunfmt.

> Parsing and validating JSON as input for msgunfmt is, of course, harder


Parsing and validating a JSON of a specific schema can be done without an extra JSON library (see x-rst.c); it's only the parsing of generic JSON that would require a JSON library. That's what I inferred after looking at a couple of JSON libraries for C.

> we only deal with strings that are trivial to encode


Still, we should deal with LINE SEPARATOR and PARAGRAPH SEPARATOR in such a way that the resulting JSON can be embedded in JavaScript (so that the client can use eval to parse it).

Bruno Haible <haible>
Group administrator
Mon 05 Feb 2018 04:11:45 PM UTC, original submission:  

(G)MO files contain all information needed for message translation at runtime in a very compact form.

Today, the JavaScript Object Notation JSON could be used as an alternative format.  Benefits would be:

- It is not binary.
- JSON parsers are universally available
- JSON is almost as compact as MO.
- It is to a certain extent human-readable.
- It uses Unicode.

The last feature could also be considered a problem but I think for practical purposes, limiting the encoding to UTF-* is rather an advantage today.

The "schema" would be straightforward.  JSON translation catalogs should probably have this form:
 
    {"Hello, world!":["Bonjour le monde!"]}

The values should be arrays, so that plural forms can be represented.  While parsing binary data in JavaScript is cumbersome, splitting a key at a possible EOT into a context and a message id is simple:

    var parts = key.split("\004", 2);

These JSON files could be easily converted into just about any other format with a similar notion of a translation database.

But especially for JavaScript, this improvement would be very helpful.  At the moment, there is a large variety of translation frameworks based on PO files for JavaScript.  They are based on PO files because the binary MO format is hard to parse.

These frameworks almost exclusively use either PO files directly, or parse the PO files and convert them to JSON.  This is inefficient, error-prone, and you lose important msgfmt features like '--check' and '--statistics'.  An new msgfmt option '--json' would make these hacks unnecessary.

Writing JSON is very simple in this case because we only deal with strings that are trivial to encode.  That means, it would not pull in any external dependencies.

Parsing and validating JSON as input for msgunfmt is, of course, harder but also less important.

Just in case: this feature request has nothing to do with https://savannah.gnu.org/bugs/?48903

Guido Flohr <gflohr>
Group Member

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by nihilus (Posted a comment)
  • -email is unavailable- added by haible (Posted a comment)
  • -email is unavailable- added by gflohr (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

     

    No changes have been made to this item

    Back to the top

    Powered by Savane 3.13-3230.
    Corresponding source code