bugFree UCS Outline Fonts - Bugs: bug #38735, Make pnum, onum, lnum, tnum, zero,...

 
 

bug #38735: Make pnum, onum, lnum, tnum, zero, frac OT features work

Submitter:  Emmanuel Vallois <emmanuel_vallois>
Submitted:  Sun 14 Apr 2013 02:37:54 PM UTC
   
 
Category:  character range Severity:  3 - Normal
Item Group:  character substitution issue Status:  Fix posted
Privacy:  Public Assigned to:  Stevan_White
Open/Closed:  Open Release:  * development
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Thu 13 Jun 2013 07:59:36 PM UTC, comment #29: 

Hey Emmanuel,

How are we doing on this report?

I've made a lot of submissions regarding the issues you raised...
What further do you think needs to be done?

Cheers!

Steve White <Stevan_White>
Group administrator
Tue 21 May 2013 09:48:40 AM UTC, comment #28: 

Hi, Emmanuel,

I'm aware of the 'tnum' issue in bold italic, as well as what triggers it (the order of the lookups).  I haven't yet analyzed why this happens, and I'll leave the problem there until I understand it.

This misunderstanding may be related to the absence of documentation stating just what level of code is responsible for activation of features (particularly the new CSS OpenType ones).

>I stumbled upon a mapping error in Unicode


Another Unicode note: the reverse capital R is not a capital form
of the reverse r. They're separate symbols.  I don't know how this distinction is indicated in Unicode.

>Yes, the four features are mutually exclusive two by two. Look at the first
>table, and see the first column (lining) is exclusive with the second
>(old-style), and the first row (proportional) is exclusive with the second
>(tabular).


Besides the current behavior of Firefox, on what authority do you say this?

>>I have doubts about changing a tnum to a pnum and then back again. For one
>>thing, it suggests a loop.  I have consciously avoided such loops in the past.
>> (Of course, the font being data can't have loops -- but the question is, when
>>should the renderer stop transforming?)
>

I regret making the comment about loops.  I was just freely writing my thoughts, trying to understand what my objection is here.  Unfortunately it was noise in this conversation, that only produced a lot more noise.

But you guys really need to explain your reasoning when you make your pronouncements.  I will never accept a pronouncement without substantiation. Maybe I'm dull, but that's how it is.

I am impressed by your experiment.  This still shows only the behavior of particular renderers, but I tried it in several environments, and found it consistent.

However I don't know where in any documentation at my disposal, it says just how these things are supposed to work.  That would be very helpful.

>So either make these Latin-language-only, or keep the status quo.
>I thought that dlig would be applied on large chunks of text, because I was
>thinking about ligatures such as st, but in FreeFont these are in hlig, so
>there is no conflict. But then what about the tz ligature? You probably know
>more about it than me.


Do I understand you to say, you accept my assessment, and that it's OK to leave the oe-ligature in the 'dlig'?

The tz is a stylistic ligature in German, once used regularly,
still occasionally seen in titles and signs.  (See Wikipedia.)

>>11) regarding Cyrillic, I don't understand your remarks "Still lacks".


>This is by comparison to the regular face, where such substitutions are made.
>I assume here that the regular face is the model to be followed. Since
>moreover these don’t imply creating new glyphs but simply adding new entries
>in the lookup subtable, I thought it was straightforward.


OK, you're referring to the fact that the substitutions are different for italic and regular.

Yeah, I guess I lost steam there--I made a couple more (partly guessing).

Be aware, the regular forms of several of the alternate Bulgarian
style are simply styled after the italic forms -- there is thus no special italic form.  I added the kje as a k-accent for consistency, although the letter isn't used as such in Bulgarian.


Steve White <Stevan_White>
Group administrator
Tue 21 May 2013 12:26:34 AM UTC, comment #27: 

With r2587, this bug (as far as its original scope) is now fixed. Hmm… except in FreeSerif Bold Italic. There, the tnum feature doesn’t work as expected. The table is identical to that in other faces though. The difference lies elsewhere… the relative order of the {o,p,t,l}num tables. When moving up the pnum table by two position to have the same order as in other faces, magically it works!

Thanks Steve for these working tables, and the bonus zero and frac (with all fractions from halves to tenths)!

So let’s move on to continuing work on other OT features that was spurred by this bug, see bug #39029 “subs, sups, smcp and c2sc OT features”.

Cheers!

Emmanuel Vallois <emmanuel_vallois>
Wed 15 May 2013 06:22:38 AM UTC, comment #26: 

Only a small remark about loops: there’s no need to fear loops in opentype features. I had this explained to me once by a guru (don’t know if from MS, Adobe or freesoftwareworld). Sadly I can’t find that conversation, so I can only tell you in my own words.

On a glyph the engine tries to match lookup by lookup in the order they appear in the font. If it finds a match it’s applied and doesn’t look any further in this lookup and moves on to the next (→ only the first match in a lookup gets applied). After going through all lookups, the pointer moves to the next glyph where the same procedure takes place. Thus every lookup gets applied only once per glyph, there’s no going back to a previous one and no repitition. Also, there’s no going back if a feature contains a lookup that is positioned earlier in the list of lookups. The lookup will always be applied in the order of appearance and only once.

This means you could switch between two variants of a glyph in every lookup in a font, but for creating a loop you’d need to build a font with an infinite number of lookups.

Georg Duffner <georgd>
Wed 15 May 2013 05:26:18 AM UTC, comment #25: 

Steve,

Well done! Thanks for sending the PDF, it’s really nice to see Devanagari &
Bengali features working.
All my comments are addressed, except 6) (see below) and b) (about music,
the calt deals with articulation marks discussed in bug #26801, postponed).
And Thai works perfectly, thanks.

I have uploaded my usual test file updated with a bunch of new comments.

While examining the newer test file (The Big Table), I stumbled upon a
mapping error in Unicode, where the superscript turned Latin epsilon
(U+1D4C … OPEN E) decomposes to reversed Latin epsilon (U+25C REVERSED
OPEN E), which has its proper superscript at U+1D9F. To see the problem,
look at the lines 025C and 1D08 of the updated version of The Big Table.

>> 6) There is no lnum table. The _x_num features may be applied
>> one after the other, so it still makes sense to have one. It
>> is no more than above a table to map everything to default.
>> Its mappings are from old.X to X and from oldprop.X to fit.X.
>> The last version of my test already includes a test for that.
>
>I have my doubts about this.  I have to think it through.
>
>The problem has to do with the layers of code, and who is responsible for
>what.  Of course the bottom layer is the font, (which is just data), and
>specifies what needs to be done to make a proportional number, etc.
>
>Does your new test show it misbehaving somehow?

Yeah, the last column of the second table should be identical to the first.

>Does proportional mean not tabular?  Are they exclusive?


Yes, the four features are mutually exclusive two by two. Look at the first
table, and see the first column (lining) is exclusive with the second
(old-style), and the first row (proportional) is exclusive with the second
(tabular).

>I have doubts about changing a tnum to a pnum and then back again. For one
>thing, it suggests a loop.  I have consciously avoided such loops in the past.
> (Of course, the font being data can't have loops -- but the question is, when
>should the renderer stop transforming?)
>

There’s no loop problem as these features are only applied on-demand. The test
in the second table is precisely doing a loop, first row in clockwise order
viz. the first table, the second row in counterclockwise order. The letters in
the headers tell which features are applied (first letter of feature name),
and in which order. The renderer applies the features in order, and stops after
doing one substitution for each feature applied. So for example, when in the
second table it says “Tab. old-style pot”, this means tabular old-style by
applying, in order, pnum, onum and tnum. The first substitutes X with fit.X
(pnum), then fit.X with oldprop.X (onum), then oldprop.X with old.X (tnum).

It is not useful here, as only onum would have sufficed, but if a style was
applied that specified proportional old style, and on top of that another
specified tabular numbers, the tnum table would still be useful, despite
tabular being the default width for numbers.

There is also no risk of looping because only half the glyphs are mapped in each
lookup table. Applying tnum or lnum on the default numbers would have no effect,
because those would not appear as source glyphs in these tables.

To bring back my point, the lnum table is as useful as the tnum table, no more,
no less, so I was surprised that you kept the tnum but ditched lnum.
lnum can, as above, be useful if the general style has been set to old-style
numbers, but at certain places (e.g. a table, as I think lining numbers are most
likely to be seen there) we want to switch back to lining numbers.

With the completion of the lnum table, the original purpose of the bug will be
fulfilled. The discussion has drifted on other features since then.

>10) About the oe ligature (also ae).
>[…] in English, until the mid 20th century, oe was in fairly common
>use in words of Latin or Greek origin (I doubt how consistent this ever was).
>Now it's much rarer, and would be considered a stylistic thing, and therefore
>appropriate for dlig.

I would say the same for Latin-language words in French, œ and æ were then used,
but nowadays Latin words are mostly written without ligatures, but they may be
used here and there.
So either make these Latin-language-only, or keep the status quo.
I thought that dlig would be applied on large chunks of text, because I was
thinking about ligatures such as st, but in FreeFont these are in hlig, so
there is no conflict. But then what about the tz ligature? You probably know
more about it than me.

>11) regarding Cyrillic, I don't understand your remarks "Still lacks".

This is by comparison to the regular face, where such substitutions are made.
I assume here that the regular face is the model to be followed. Since
moreover these don’t imply creating new glyphs but simply adding new entries
in the lookup subtable, I thought it was straightforward.

>As to the x-height of Bulgarian style Cyrillic sha, I looked at it.
>It's the same as all the others, 461 EM.  What are you looking at?
>If you are referring to the rendering... I doubt there's much to be done.

Sorry about that, autohinting was enabled when I generated the font… my
fault.

(file #28079, file #28080)

Emmanuel Vallois <emmanuel_vallois>
Tue 14 May 2013 07:57:03 AM UTC, comment #24: 

Emmanuel,

See my PDF taken from FreeSerif on Linux.  It's mostly all working to my satisfaction, except the Music range.  Serif bold face will be a lot of work.  I'm not sure about some of the small caps, and I have yet to investigate your remarks about Arabic and Thai.

> 1) the subs and sups tables and the frac entries for non-Unicode
> diagonal fractions in FreeSans Regular were deleted in r2561;


Well, they're there now.

> 2) in FreeSerif Bold, hlig,smcp,c2sc,subs,sups still don’t work for
> e.g. Catalan and German, or Turkish, because it only has latn{dflt}.
> For subs and sups also in Regular.


I think I got them all.  I see I wasted some hours already because I hadn't seen yet another fontforge bug.

> 3)The subs and sups in Bold contained entries for numbers, I thought
> this were to be generalized to Regular face, but instead they were deleted.


I hadn't noticed.  I have attempted to re-instate them.

> 4) The “c2sc for Turkish” table is useless (all FreeSerif)


OK this is removed now.

> 5) The tnum table is still incorrect, it is currently implemented as a “map > everything to default tabular lining numbers”, which it is not made for. It
> should only map fit.X to X and oldprop.X to old.X. The last version of my
> test already includes a test for that.


Right. done

> 6) There is no lnum table. The _x_num features may be applied
> one after the other, so it still makes sense to have one. It
> is no more than above a table to map everything to default.
> Its mappings are from old.X to X and from oldprop.X to fit.X.
> The last version of my test already includes a test for that.


I have my doubts about this.  I have to think it through.

The problem has to do with the layers of code, and who is responsible for what.  Of course the bottom layer is the font, (which is just data), and specifies what needs to be done to make a proportional number, etc.

Does your new test show it misbehaving somehow?

Does proportional mean not tablular?  Are they exclusive?

I have doubts about changing a tnum to a pnum and then back again. For one thing, it suggests a loop.  I have consciously avoided such loops in the past.  (Of course, the font being data can't have loops -- but the question is, when should the renderer stop transforming?)

> 7) in FreeMono other than Regular, zero will only work when
> not in {Latin, Cyrillic, Arabic, Hebrew}, in Regular, only
> Cyrillic (don’t forget Serbian and Mcedonian) is excluded.
> In FreeMono, frac will not work in Cyrillic.


I've taken care of some of these details, and written a script to automatically check for this kind of problem.

> a) stylesets 2, 3 and 4 for Devanagari still don’t work in my browsers,
> but I think that FreeFont is not to blame for that.


Not directly... Windows is turning this off for unknown reasons.
But Windows is doing several other unfortunate things, too.

> b) on calt, it replaces (musical) combining signs with spacing
> variants, which in itself can make sense, what baffles me is the
> nonsensical context. It would maybe make sense to change the
> context to be “when following a space/no-break space”. To provide
> the user with alternatives, there is already the aalt table.


A) There's an old open bug report on this range.
   https://savannah.gnu.org/bugs/?26804
   Could we move this discussion there?

B) the whole music range has almost never drawn comments.
   I'm very glad to hear yours!
   I think this was an experiment with the Music range.  It doesn't make sense to me either.  My guess is, something went wrong with it...
   (I mean to check the revision history)
   But I still want to implement something with the range.

C) I had no idea what to do with the obvious combining forms in this range.  I imagined symbols appearing in-line in text, and thought, how might they profitably be combined.  I know of no documentation.  I have my doubts that it all makes sense in Unicode.

D) I don't really understand most of your remarks here.  I'm sure much of what I did doesn't make sense, but I think it's sort of a blank slate--any sense is yet to be made.  I am glad to hear your ideas

> 8) superscript latin epsilon and iota:


   These phonetic symbols were meant to map from lowercase Phonetic  symbols, not Greek letters.  Fixed.

9) the Bengali fractions do work on Linux

10) About the oe ligature (also ae).

I take your point about the oe in French.

By contrast, in English, until the mid 20th century, oe was in fairly common use in words of Latin or Greek origin (I doubt how consistent this ever was).  Now it's much rarer, and would be considered a stylistic thing, and therefore appropriate for dlig.

It would be possible to separate a table for this special use in English, but I have reasons to avoid that.

That said, I wonder if we're misjudging how these tables ought to be applied.

Except for the required and standard ligatures, the other tables ought not be normally be turned on globally in a document.  Rather, they should be applied carefully to individual groups of letters.

The case is similar for the long s, and for the ff, ffi etc ligatures in German.

Think of it this way: use the "discretionary" table --locally-- if you mean to add a ligature as style -- otherwise, don't use it.

In the case of French, it would be better not to use the discretionary table at all, and to encode the ligatures explicitly.

In English, for example, a style class "latinname" could be applied to Latin species names, automatically implementing the oe ligature just for these names.

The exact application of such features are hinted about in the OpenType docs, but really mostly it's left to the developer.

11) regarding Cyrillic, I don't understand your remarks "Still lacks".
As to the x-height of Bulgarian style Cyrillic sha, I looked at it.
It's the same as all the others, 461 EM.  What are you looking at?
If you are referring to the rendering... I doubt there's much to be done.


Cheers!

(file #28077)

Steve White <Stevan_White>
Group administrator
Sun 12 May 2013 09:47:27 AM UTC, comment #23: 

Here is a new test that checks all Latin characters (and a few others) for sub, sup and small-caps, giving for comparison the Unicode character when it exists. I used Unicode decomposition mappings, plus a few that do not have a decomposition, plus small-caps mappings done by hand with all Unicode characters that have “SMALL CAPITAL” in their name.

With the information from this script generated file, I updated my former test with all the characters I came up with.

The contextual alternates test doesn’t make sense because the table in the font doesn’t make sense. I was basing the tests on all “optional” features I found present in FreeSerif. For this it’s useless and there is already an ‘aalt’ table to access the same glyphs, so delete it.

From the OpentType feature registry, the definition of ‘dlig’ is “Replaces a sequence of glyphs with a single glyph which is preferred for typographic purposes. This feature covers those ligatures which may be used for special effect, at the user's preference.” As I understand it, it is for purely stylistic/decorative effect, they are not required for reading the text or for to the meaning, but adding them is harmless and won’t change the text. Good examples are the st and ct ligatures in Latin. However, the combinations you’ve put here are not decorative, they are orthographic. I’m sure at least in the oe case, as it is used in my native language. Making ct and st ligatures do not change the perceived identity of the text, ligating o and e does. In French, œ is not a fancy ligature, in some words it is required, in others it is disallowed. So I think oe->œ is not a good candidate to be in the dlig table.

There is no more comments on Thai, and on Devanagari, just it doesn’t work for me. I think the renderer on Windows 7 just doesn’t apply non Devanagari-specific features in Devanagari, so hlig and ss0x don’t work. Or I should restart the system…

For the Arabic ‘liga’ ligature, I discovered it while searching for every ‘liga’ in FreeSerif. It is a feature that replaces two combining marks, in one order only, by a glyph that is looking very similar to the two combining marks’ normal placement, but they look a little farther apart, this is barely visible. In the test for each combination of marks, I put them once in the wrong order to show the unligated behavior (that should appear first, on the right), then the combinations in the correct order to show the ligature. Maybe the ligature is to mimic the reordering of the marks, for the text to look correct whatever the order of combining marks is.

I also have an idea about making small-caps (and also sub/sup) work for accented characters. But I will file a separate bug on that later.

(file #28067, file #28068, file #28069)

Emmanuel Vallois <emmanuel_vallois>
Tue 07 May 2013 04:16:14 AM UTC, comment #22: 

Hi,

I'm working on a point-by-point, but I won't be able to work on this for the next day or so.

A couple of things:

You haven't yet responded to my questions about the Contextual alternates tables, so I still don't know what you expect these tables to do in a web browser.

I'm not sure I have the exact test page you are using.  As to what you might put in it -- as a suggestion:
I had in mind: rather than listing sup/sub characters that
happen to be in FreeSerif, pull them instead from the UnicodeData.txt file. And the Contextual alternates test doesn't make sense I can think of.  And the comment about ae and oe, and about Devanagari, and about the Thai ligature.  And I don't understand the comments about the Arabic marks.

Yeah, let's keep plugging at it!

Steve White <Stevan_White>
Group administrator
Tue 07 May 2013 02:09:08 AM UTC, comment #21: 

Wow!

Already changes arrive!
Points 2, 3, 5 addressed.
7 mostly. In FreeMono Italic, Bold Italic, and Bold, there’s still not Cyrillic. And elsewhere too (FreeSans, FreeSerif), frac, xnum, zero, subs, and sups do not work with Cyrillic.

And bravo for implementing superiors and inferiors that were absent from FreeSerif Italic faces.

I didn’t think this bug would finally go this far, but we’re now nearly there, and it’s nice! Big up!

Emmanuel Vallois <emmanuel_vallois>
Mon 06 May 2013 05:39:10 PM UTC, comment #20: 

Steve,

It’s far batter now! You’ve done a great job, but as all these script language combinations are not always easy to spot, here are some remarks on what I spotted But
1) the subs and sups tables and the frac entries for non-Unicode diagonal fractions in FreeSans Regular were deleted in r2561;

2) in FreeSerif Bold, hlig,smcp,c2sc,subs,sups still don’t work for e.g. Catalan and German, or Turkish, because it only has latn{dflt}. For subs and sups also in Regular.

3)The subs and sups in Bold contained entries for numbers, I thought this were to be generalized to Regular face, but instead they were deleted.

4) The “c2sc for Turkish” table is useless (all FreeSerif)

5) The tnum table is still incorrect, it is currently implemented as a “map everything to default tabular lining numbers”, which it is not made for. It should only map fit.X to X and oldprop.X to old.X. The last version of my test already includes a test for that.
6) There is no lnum table. The _x_num features may be applied one after the other, so it still makes sense to have one. It is no more than above a table to map everything to default. Its mappings are from old.X to X and from oldprop.X to fit.X. The last version of my test already includes a test for that.

7) in FreeMono other than Regular, zero will only work when not in {Latin, Cyrillic, Arabic, Hebrew}, in Regular, only Cyrillic (don’t forget Serbian and Mcedonian) is excluded.
In FreeMono, frac will not work in Cyrillic.

a) stylesets 2, 3 and 4 for Devanagari still don’t work in my browsers, but I think that FreeFont is not to blame for that.

b) on calt, it replaces (musical) combining signs with spacing variants, which in itself can make sense, what baffles me is the nonsensical context. It would maybe make sense to change the context to be “when following a space/no-break space”. To provide the user with alternatives, there is already the aalt table.

There’s nearly no change needed to my test file, but I’m already thinking about another form of test.

Watch this space!

Emmanuel Vallois <emmanuel_vallois>
Sat 04 May 2013 12:45:12 AM UTC, comment #19: 

Emmanuel,

Have a look at your test again with the latest commits (I would be glad to see an update of that!)

Regarding the contextual alternatives... I don't understand how this was supposed to work.  My understanding is that all these alternatives features were intended to provide the user with selection of possible alternatives, which they would choose via the application (on a case-by-case basis).  I don't understand what "activating" such a feature would mean.

Cheers!

Steve White <Stevan_White>
Group administrator
Fri 03 May 2013 06:19:47 PM UTC, comment #18: 

Emmanuel,

I just posted some changes to FreeSerif that have bearing on this report: at least, you'll see the Devanagari style sets working now.  Thanks to Georg!

The other issues may be related -- I'm not sure.

Steve White <Stevan_White>
Group administrator
Thu 25 Apr 2013 12:57:22 PM UTC, comment #17: 

Hi Emmanuel,

> What I said about this last item in [#comment12] has not be
> catered for in the following tables as of r2546 (in FreeSerif,
> all weights):


Not sure what you're referring to.  That was a long comment,
covering several subjects -- and this whole thread has become very confused. 

In particular, the question of how features ought to be
activated by script/language remains in dispute.  I still
regard it as a bug in rendering software, but I am researching
that, and will discuss it in the thread Georg opened when I
have more information.

Is this what was not "catered for", or did you mean something else?

> onum table lacks fit.X→oldprop.X;


fixed.

> tnum is wrong: this table is only about width, not old vs.
> lining; so old.X→X entries should be removed, and oldprop.X
> should map to old.X, that is, for both styles, mapping from
> variable to fixed-width i.e. tabular;


I see fit.X -> X entries in this table, which is right.
Please look again.

But anyway... given the numbers are "tabular" to begin with, they could only become "fit" if a "pnum" had previously transformed them, in which "tnum" would conflict...
I'm removing the table.

> lnum is wrong: this table is only about style, not width;
> so fit.X→X entries should be removed, and oldprop.X should
> map to fit.X, that is, for both widths, mapping from old-style
> to lining.


I removed the table.  My understanding is, the only way an old-style figure could match this table is if 'onum' had been activated, which would conflict with 'lnum' anyway.

> And in FreeSerif italic, check that “'pnum' proportional-width
> numbers-1” indeed maps to oldprop.{zero,one,two,…} instead of
>oldprop.{0,1,2,…}. This may be a leftover of the glyph
> renaming. Or is my working copy corrupted?


That's scary.  I blame this one on FontForge.  Near as I recall, I only changed the names as I did in the other faces, which should have changed the names in the tables.

Anyway, it's fixed.

Steve White <Stevan_White>
Group administrator
Thu 25 Apr 2013 01:47:37 AM UTC, comment #16: 

The original patches I submitted in this bug dealt with to things:

  • Feature activation
  • The contents of the onum, pnum, lnum, tnum tables.


What I said about this last item in [#comment12 comment #12] has not be catered for in the following tables as of r2546 (in FreeSerif, all weights):

  • onum table lacks fit.X→oldprop.X;
  • pnum is correct;
  • tnum is wrong: this table is only about width, not old vs. lining; so old.X→X entries should be removed, and oldprop.X should map to old.X, that is, for both styles, mapping from variable to fixed-width i.e. tabular;
  • lnum is wrong: this table is only about style, not width; so fit.X→X entries should be removed, and oldprop.X should map to fit.X, that is, for both widths, mapping from old-style to lining.


And in FreeSerif italic, check that “'pnum' proportional-width numbers-1” indeed maps to oldprop.{zero,one,two,…} instead of oldprop.{0,1,2,…}. This may be a leftover of the glyph renaming. Or is my working copy corrupted?

Emmanuel Vallois <emmanuel_vallois>
Mon 22 Apr 2013 08:34:07 PM UTC, comment #15: 

Emmanuel,

I had missed your remarks -- I spent a big part of the day re-checking complaints from other people.

I fixed the Turkish dotted I last night, as I'm sure you saw in the logs.  I tested it in a couple of places, but not everywhere.

Are you referring to my fix, or something before the fix?

I'm aware of the OpenType documents, thanks.

It will really help to kepp the discussion on-topic.

Steve White <Stevan_White>
Group administrator
Mon 22 Apr 2013 06:21:36 PM UTC, comment #14: 

Georg,

Thanks for the advice!  Since they have become available (in the past couple of years), we do test under Uniscribe and Harfbuzz.  And we test under a dozen applications on multiple versions of multiple OS's.  And -- regarding most the features you mentioned, as near as I can tell, they're working right now.  I'm looking right at them with my own eyes.

Try it for yourself.  If you have specific bugs to report, please do so in an appropriate thread, with the necessary document attached for us to address them.

Again, this thread is about the new features being tested in web browsers.  You seem to be complaining about how we implemented feature activation -- which isn't really the same thing.  It would be best to keep the issues separate.  Please feel free to open a separate bug report!

Steve White <Stevan_White>
Group administrator
Mon 22 Apr 2013 06:06:57 PM UTC, comment #13: 

Steve,

I didn’t test, I read the source. I looked at the Free Serif sfd from current trunk (and extracted a feature file from it, because I prefer to read features that way). To make sure that a lookup has a chance to work in the wild look at it in the metrics window in  FontForge. What does not work there, won’t work anywhere.

When testing, you should take reliable engines as a reference, which are Uniscribe (I don’t have access to it atm) and harfbuzz-ng. The problem with them is, one usually tests them in other applications which rises the probability of errors. E.g. for the language tags you can’t test with Chrome because it uses unvariably the Browser’s locale, not the lang defined in the html document. Firefox has proven to be more reliable, the upcoming XeTeX is good too.

Georg Duffner <georgd>
Mon 22 Apr 2013 02:05:40 PM UTC, comment #12: 

Sorry for the noise with italic. As I generate font by hand with FontForge GUI (as I think the FontForge for Windows I have doesn’t have Python scripting), and I was not using the same flags, I regenerated the fonts with the correct flags and that works!

I don’t know why but the “Apple” checkbox was checked, and I think that messed up the generated files to be partially unusable on Windows.

Another thing, that, as of r2543, is still not fixed, is the onum pnum lnum tnum tables themselves.
If have the table of my test file in front of you:
*onum should map each digit in the first column to the corresponding digit of the second column (fit.X→oldprop.X, X→old.X)
*lnum should be the reverse mapping (oldprop.X→fit.X old.X→X),
*pnum should map each digit in the second row to the corresponding digit of the first row (X→fit.X, old.X→oldprop.X),
*and tnum should be the reverse mapping (fit.X→X, oldprop.X→old.X).
No more, no less. Currently some tables are correct, some have to much entries, some do not have enough.

To confirm Georg’s saying, try to apply smcp or c2sc in turkish to the whole alphabet, only the i’s will be in small-caps, because only the turkish-specific table is triggered (latn{TRK ,…}), and not the general table (marked latn{dflt}). To have it work, we’d put latn{TRK , …, dflt} to the general table.

The logic of functionality grouping is defined in OpenType™ Layout Common Table Formats.

The features are grouped by script, then by language, and form a tree. As I understand it, only features attached to a single branch of the tree will be applied at once on a given run of text.
Example:
FreeSerif
├ latn
│  ├ 'CAT '
│  │  └ 'liga' std. ligatures in Latin for Catalan
│  ├ 'TRK '
│  │  └ 'smcp' lowercase to small caps in Latin for Turkish
│  ┊
│  └ 'dflt'
│      ├ 'liga' std. ligatures in Latin
│      ├ 'smcp' lowercase to small caps in Latin
│      └ 'onum' oldstyle figures
├ cyrl
│  └ 'dflt'
│      └ 'onum' oldstyle figures
├ grek
│  └ 'dflt'
│      └ 'onum' oldstyle figures
└ DFLT
    └ 'dflt'
        ├ 'frac' diagonal fractions
        └ 'onum' oldstyle figures
(This is an extract of current FreeSerif Normal)

Here, if text is marked as Catalan, it will take the latn/CAT branch of the tree and apply the Catalan-specific ligature (l·l) but not the common “std. ligatures” which are on the latn/dflt branch (I tested and it works that way in Firefox and IE10).

Same goes with Turkish text when we apply smcp to get small-caps. The only letters that get small-caps are the i’s that are specially treated in the “lowercase to small caps in Latin for Turkish” on the latn/TRK branch, and the other letters are not modified, as the general “lowercase to small caps in Latin” is not on the same branch (it is on latn/dflt).

And for onum, as defined here they will work in latin text that is not Catalan, not Turkish, and not another language that gets special treatment, in Cyrillic but not in Serbian, Macedonian, and Bulgarian that get special treatment, in Greek (script), and in any other script that does not appear elsewhere.

Emmanuel Vallois <emmanuel_vallois>
Mon 22 Apr 2013 01:10:50 PM UTC, comment #11: 

George,

Also, in this bug report, Emmanuel and I are discussing the functionality of the font regarding the new OpenType property features being tested in new browsers.  Your complaints do not seem to be on that topic. 

It would be best if you open a new bug report (containing the information I requested in my previous posting).  Or you can write to me directly at my personal address, which you'll find on the FreeFont project pages.

Thanks!

Steve White <Stevan_White>
Group administrator
Mon 22 Apr 2013 12:01:34 PM UTC, comment #10: 

Hi Georg,

There seems to be be some confusion.

> Upon deeper inspection, this bug concerns all OT features! It concerns GSUB
> and GPOS features equally.
>
> Thus, for example there’s no kerning or mark positioning for german,
> catalan, spanish, dutch, turkish and sami.
>

I'm looking right at it in Windows 7 and in Linux, on several browsers. Kerning and mark positioning are working properly in all of them, in German, Caatalan, Spanish, Dutch, Turkish and Sami.

We have a lot of test pages, and I have personally checked them all.  Surely I make mistakes, but nothing like what you're reporting.

Is it possible that you are looking at WebFonts?  I am aware that on some systems, many font features are turned off -- kerning in particular.
This is not a FreeFont bug -- it is a bug on those systems.

If you look at my previous attachment, at least some of the prob;ems Emmanuel reported are not showing up on my systems.  We are investigating why that may be so.  (Your input too will be taken seriously!)

** Please describe:
    1) your operating system
    2) the application program you are using
    3) the document you are looking at
    3) the version of FreeFont you are referring to
** Please also include a copy of any documents you are viewing,
and say explicitly what problem you perceive with the output.

> In indic scripts you have for example feature akhn for dev2{dflt}, dev2{GUJ },
> dev2{SAN } and dev2{SAT }. Feature blwf has only dev2{dflt} and dev2{SAN }.
> Thus there’s no feature blwf for SAT and GUJ. In fact, akhn is the only
> feature defined for GUJ and SAT,


Let's try it.  Here is some Gujarati text written in Devanagari script
   http://en.wikipedia.org/wiki/Gujarati_language
(see attached screenshot).
Several Devanagari GSUB features can be seen functioning in that picture.   It appears that you are mistaken as to how feature activation works, at least on this system.

** Please give us a link to a document that describes the
logic of feature activation as you understand it, and please indicate what passage(s) in the document have been violated by our fonts.

> so all the others won’t be applicable for
> these two languages which might result in unusable output (just guessing as I
> don’t know these languages, but as far as I know indic writing systems, this
> is probably very buggy).
>

The Devanagari range has been tested by speakers of several of these languages. The 2012 release was working with extant versions of Firefox and XeLaTex, on Windows 95 and Linux.  Hindi and Sanscrit were heavily tested, and pages of real-world text in several other languages were reviewed in several environments, and found to be flawless, before that release of the software.

Now, some things have changed regarding Indic ranges in Windows Vista and 7.  Some features of the 2012 releas don't work properly on those systems.  We have been working hard to remedy that.  But these don't seem to be the problems you're referring to.

Table activation is terribly complicated -- the logic is often not what one expects. 

* Please provide an example of incorrect *rendering
   (screenshot and original document, please!)

There surely are some bugs, and we want to fix them.
Let's work together to sort those out!


Steve White <Stevan_White>
Group administrator
Mon 22 Apr 2013 08:53:31 AM UTC, comment #9: 

Upon deeper inspection, this bug concerns all OT features! It concerns GSUB and GPOS features equally.

Thus, for example there’s no kerning or mark positioning for german, catalan, spanish, dutch, turkish and sami.

In indic scripts you have for example feature akhn for dev2{dflt}, dev2{GUJ }, dev2{SAN } and dev2{SAT }. Feature blwf has only dev2{dflt} and dev2{SAN }. Thus there’s no feature blwf for SAT and GUJ. In fact, akhn is the only feature defined for GUJ and SAT, so all the others won’t be applicable for these two languages which might result in unusable output (just guessing as I don’t know these languages, but as far as I know indic writing systems, this is probably very buggy).

Georg Duffner <georgd>
Sun 21 Apr 2013 02:28:55 PM UTC, comment #8: 

Regarding your images:

Chrome italic
=============

Latin letters look like FreeFont, but with no features turned on.
Arabic and Thai letters are not FreeFont.

Firefox 2.1 italic
==================
Latin is like Chrome case.
Arabic... looks like slanted FreeSerif
Thai looks like FreeSerifItalic, but with features turned off.
Curiously, the Hanonoo features are working!

IE
==
Arabic isn't FreeFont
It's a mess much like the others.
   ============================================================
Here's what I see with your latest file "Features text.html",
on Firefox 20.0.1 on Windows 7.
All the ranges come from FreeSerifItalic except Arabic which is coming from FreeSerif. (This is a fairly recent version of the fonts, but not from the last few commits.)
As you see most of it is working quite well.  Nothing like the problems you're seeing.

Possible causes:

1) Recent commits?  Doubt it -- haven't done much with italic lately.

2) Something related to Webfonts? 

3) Is it possible you have junk versions on your system?
   In a command shell, look in C:\Windows\Fonts\ 
   dir Free*

Cheers!


Steve White <Stevan_White>
Group administrator
Sun 21 Apr 2013 01:38:17 PM UTC, comment #7: 

Hi Steve,

This is not a Firefox bug but a bug systematically applied in the font, because of the way languagesystems are supposed to work in OT fonts.

Once you define a languagesystem, it gets excluded from DFLT{dflt} for the whole font. If you want a feature to be applicable to every languagesystem you have to inlcude every single system which you use in the font into the feature.

Also, they get excluded from the script default (like latn{dflt})!

So in this very case, the lookups may only work with languagesystems not defined, e.g. cherokee.

These lookups need not only latn{dflt} but also cyrl{dflt} and grek{dflt} (and probably more of this kind) but also (as you define localized lookups for various languages) latn{DEU }, latn{TRK }, latn{CAT }, etc.

Georg Duffner <georgd>
Sat 20 Apr 2013 09:39:09 AM UTC, comment #6: 

Hi Emmanuel,

I have confirmed that including "latn{dflt}" in the list of languages to trigger the 'pnum' feature causes it to function (in Latin-based languages) in Firefox, whereas with "DFLT{dflt}" it does not.  This is not right.

This feature is not at all language-specific.  It is meant to be turned on by the application, not by the language.  Likewise with the other features you mention. It is wrong for the font to list every script in which the feature might be turned on -- DFLT{dflt} is the proper trigger for the feature

This as a Firefox bug.  Will you be reporting it?


Steve White <Stevan_White>
Group administrator
Wed 17 Apr 2013 08:27:54 PM UTC, comment #5: 

Hi again Emmanuel,

I haven't been able to determine your normal e-mail address.  You should have mine though--the gmail one.  Let's talk about the other features there.

I have been trying to get your test pages to work on Windows 7, using Firefox there.  Do you know if this should be possible?

The reason is, we want to sort out who is responsible for what.  The new Harfbuzz is supposed to emulate the Windows font rendering.

We do not want to build in a fix for a bug in some other software.  This can confuse issues terribly.

Steve White <Stevan_White>
Group administrator
Tue 16 Apr 2013 11:33:17 AM UTC, comment #4: 

Emmanuel,

Regarding the rest of your superb HTML tests, let's dissucss it off-line, to determine which are font bugs...

Steve White <Stevan_White>
Group administrator
Mon 15 Apr 2013 10:47:48 PM UTC, comment #3: 

Hi Emmanuel.

The logic of table activation is weird and surprising.  I only know what I have observed:

If the table is supposed to be activated for SCR1(dflt), and no other language is specified, then it is activated for any language using script SCR1.  However if there are two languages lng1 and lng2 useing script SCR1, and the table is set to activate for
     SCR1(dflt), SCR1(lng1)
then it will not activate for lng2.

I know this is very counter-intuitive.  (Again, maybe I'm wrong.)

Please try this on your system, and tell us what happens:

Declare a run of text to be Russian-language.
Put a digital number in the run of text, and try to invoke these special features.

Steve White <Stevan_White>
Group administrator
Mon 15 Apr 2013 04:02:45 PM UTC, comment #2: 

Hi Steve,

No, I didn’t ditch the DFLT{dflt}, I added latn{dflt}. I don’t understand either why this seem necessary, at least in Windows browsers, as DFLT{dflt} should be sufficient, as I understand the standard. But sometimes the standard is not sufficiently clear, and implementations vary in their interpretations of it. It wouldn’t be the first time Microsoft did things slightly their own way.

In short this addition should not result in loss of functionality for environments where this already worked, I think I wouldn’t have posted it until I found an acceptable solution. And note I’ve posted a patch for italic fonts as well. I will also test Word 2010 for the few features we can use there.

Emmanuel Vallois <emmanuel_vallois>
Mon 15 Apr 2013 10:11:17 AM UTC, comment #1: 

Hi Emmanuel,

It is surprising to me that this makes these features work in Windows.  My understanding was, DFLT(dflt) should have made it work in all script, which was my intent.

I had tested the features in XeLaTex last year, and assumed that the reason they didn't work in browsers was, they just hadn't been coded to used them.

As I understand your patch, it would make these features not work in any script except Latin.  That isn't any good for this project.

So... Am I just misunderstanding the standards?  (It happens.)  Or are we looking at a Windows bug?

Thanks so much for your observations!

Steve White <Stevan_White>
Group administrator
Sun 14 Apr 2013 02:37:54 PM UTC, original submission:  

Hi, after advancing my tests of FreeFont’s OpenType features in web browsers (Firefox 21, Chrome 26, IE 10 under Windows 7), I came up with a solution and all these work, for now on FreeSerif and FreeSerif Bold.

The problem with number styles were that the tables were not all properly set up, and overall (to my surprise!) they lacked the script latn{dflt} that made them work!

Attached are the patches to the fonts on r2528, and my HTML test page, where you’ll find the numeric features are at the end. To test the file must be along FreeSerif.otf and FreeSerifBold.otf.

Emmanuel Vallois <emmanuel_vallois>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #28080:  OT Features test.html added by emmanuel_vallois (162KiB - text/html - Updated 2013-05-15)
file #28079:  Features test.html added by emmanuel_vallois (23KiB - text/html - Updated 2013-05-15)
file #28077:  FreeSerif_test.pdf added by Stevan_White (250KiB - application/pdf - Test rendered in Firefox on Linux -- mostly working.)
file #28068:  OT Features test.html added by emmanuel_vallois (162KiB - text/html - Updated and new test, with source)
file #28069:  optional_features_test.py added by emmanuel_vallois (9KiB - text/x-python - Updated and new test, with source)
file #28067:  Features test.html added by emmanuel_vallois (20KiB - text/html - Updated and new test, with source)
file #27923:  Gujarati.png added by Stevan_White (321KiB - image/png)
file #27920:  Features-FF-20.0.1.png added by Stevan_White (46KiB - image/png - Italic working in my Windows system)
file #27874:  FreeSerif italic and bold italic - numeral styles, slashed zero and fractions.patch added by emmanuel_vallois (33KiB - application/octet-stream - Similar patch for italic and bold italic.But on these, OpenType features don’t work, at least in my browsers.)
file #27875:  Features test.html added by emmanuel_vallois (10KiB - text/html - Similar patch for italic and bold italic.But on these, OpenType features don’t work, at least in my browsers.)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by georgd (Posted a comment)
  • -email is unavailable- added by Stevan_White (Posted a comment)
  • -email is unavailable- added by emmanuel_vallois (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 18 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2016-07-02 Stevan_White StatusProceeding Fix posted
    2013-05-15 emmanuel_vallois Attached File- Added Features test.html, #28079
        Attached File- Added OT Features test.html, #28080
    2013-05-14 Stevan_White Attached File- Added FreeSerif_test.pdf, #28077
    2013-05-12 emmanuel_vallois Attached File- Added Features test.html, #28067
        Attached File- Added OT Features test.html, #28068
        Attached File- Added optional_features_test.py, #28069
    2013-05-03 Stevan_White StatusNeed info Proceeding
    2013-04-22 Stevan_White Attached File- Added Gujarati.png, #27923
    2013-04-21 Stevan_White Attached File- Added Features-FF-20.0.1.png, #27920
    2013-04-15 Stevan_White StatusNone Need info
        Assigned toNone Stevan_White
        SummaryMake pnum, onum, lnum, tnum, zero, frac OpenType features work Make pnum, onum, lnum, tnum, zero, frac OT features work
    2013-04-15 emmanuel_vallois Attached File- Added FreeSerif italic and bold italic - numeral styles, slashed zero and fractions.patch, #27874
        Attached File- Added Features test.html, #27875
    2013-04-14 emmanuel_vallois Attached File- Added FreeSerif normal - working numeral styles, slashed zero and fractions.patch, #27867
        Attached File- Added FreeSerif bold - working numeral styles, slashed zero and fractions.patch, #27868
        Attached File- Added Features test.html, #27869

    Back to the top

    Powered by Savane 3.13-f8d8.
    Corresponding source code