bugFree UCS Outline Fonts - Bugs: bug #50587, Combining half marks in FreeSerif


bug #50587: Combining half marks in FreeSerif

Submitted by:  David Corbett <dscorbett>
Submitted on:  Sun 19 Mar 2017 04:00:09 PM UTC  
Category: individual character(s)Severity: 3 - Normal
Item Group: letter positioning poorStatus: Fix posted
Privacy: PublicAssigned to: Steve White <Stevan_White>
Open/Closed: OpenRelease: development

Add a New Comment(Rich Markup)

You are not logged in

Please log in, so followups can be emailed to you.


(Jump to the original submission Jump to the original submission)

Wed 22 Mar 2017 01:21:24 PM UTC, comment #12:

Looks good. If I encounter any more letters that need table entries I’ll report them as a separate bug.

David Corbett <dscorbett>
Wed 22 Mar 2017 09:50:55 AM UTC, comment #11:


I added more table entries for Cyrillic Extended-B letters. See th e improved test, attached.

For uppercase letters, only the 2-part marks are supported so far. (with a bunch more width tables, multi-part marks could also be supported, but I don't feel like it.)

Funny thing with the U+FE2F in Latin. The positoning anchor just doesn't take, while the same anchor works for other marks and Latin, and the same mark works with the same anchor in Cyrillic. I can only conclude that some other software layer has decided the titlo is not to be used with Latin. Fine with me -- just confusing.

(file #40056)

Steve White <Stevan_White>
Project AdministratorIn charge of this item.
Tue 21 Mar 2017 10:31:59 PM UTC, comment #10:

Hi David,

I haven't impemented any uppercase letters.

The Cyrillic is now in there -- best with titlo: see attachment.

I don't know what the small gap is. This stuff is also sensitive to kerning--although that effect would be caused by a positive kern.

And I don't know what the thing with U+FE2F (over mmm) is.

(file #40055)

Steve White <Stevan_White>
Project AdministratorIn charge of this item.
Tue 21 Mar 2017 08:15:10 PM UTC, comment #9:

I still see a small gap in ⟨ϩ︤ⲛ︥⟩.

I do see the problem with U+FE2F now, e.g. in ⟨Х︮С︯⟩.

David Corbett <dscorbett>
Tue 21 Mar 2017 06:16:55 PM UTC, comment #8:

Hi David,

No attempt has been made with Cyrillic, partly because I have no examples or guidance on that. This feature is not going to be implemented for every alphabet.

Coptic: oh. I forgot the letters in the Greek range.

Please have a look at my (corrected) last upload, and see if U+FE2F doesn't badly fail to go to the left side of the preceding letter.

I'll reconsider the titlo.

Steve White <Stevan_White>
Project AdministratorIn charge of this item.
Tue 21 Mar 2017 03:33:55 PM UTC, comment #7:

The diacritics are now continuous above Latin letters, but not above Coptic or Cyrillic letters. For example, ⟨ϩ︤ⲛ︥⟩ and ⟨д︮в︦д︦ъ︯⟩ have gaps in them.

I don’t see any problem particular to U+FE2F. What are you referring to?

I do see a problem with both U+FE2E and U+FE2F, the half-titlos: they have diagonal strokes. That means that, although they touch the conjoining macron, they do so at an angle. The entire titlo should be a horizontal line with corners at the ends. See for example http://www.unicode.org/notes/tn41/tn41-1.pdf section 3.4.

Similarly, the half-tildes touch the conjoining macron at an awkward angle.

I think you attached the wrong file to your last comment.

David Corbett <dscorbett>
Tue 21 Mar 2017 03:15:19 PM UTC, comment #6:

It was vastly easier to implement in FreeMono.

Steve White <Stevan_White>
Project AdministratorIn charge of this item.
Tue 21 Mar 2017 02:35:40 PM UTC, comment #5:

I added a bunch of tables to adjust the length of the conjoining marks above lowercase Latin letters. I found this painful -- so many tables for a thing of such specialized use. Maybe some other use will arise for the tables though.

I don't know what's up with U+FE2F. I suspect a bug somewhere other than the font.

For Coptic, only the overline appears to be used. For this no change of length is necessary.

See attached test.

(file #40047)

Steve White <Stevan_White>
Project AdministratorIn charge of this item.
Mon 20 Mar 2017 09:05:50 PM UTC, comment #4:


Thanks for those docs -- that is what I was looking for. I appreciate your views; mostly I quite agree.

I made some alterations. Have a look.

I checked in Win 10: some fonts there support U+FE20-3, nothing more. The do work for putting a double-wide tilde over lettersd.
They do this by making a big mark positining lookup, one for every letter of all the Latin ranges. Good grief.

My remarks about the mark-to-mark positioning had to do with the idea that the marks could be coded following the first letter. This makes a lot of sense OpenType-wise, would be much easier to implement, and would have the advantage that the whole group of marks could be made to be of the same height as the first letter. For the record -- it was not to be.

I'm not keen on what will be required to modify the width of the connectors.
See the attachment to see how bad the problem is.

OpenType really doesn't have good facilities for this sort of thing.

(file #40042)

Steve White <Stevan_White>
Project AdministratorIn charge of this item.
Mon 20 Mar 2017 03:56:09 PM UTC, comment #3:

I’ll split my answer into two pieces: how these characters are to be used, and how the font should handle them.

== Usage ==

The Unicode Standard does not give much guidance on how to use these characters with more than two base characters. All it says is “In some limited circumstances, the combining half mark diacritics can be used in combinations to represent triple diacritics, but the display of half mark diacritics used in this way often is unsatisfactory in plain text rendering.” The characters’ proposal documents go into more depth.

http://www.unicode.org/L2/L2007/07085r-n3222r-coptic-adds.pdf section 3.3 explains the use of U+FE24 through U+FE26. The gist is that the first base character gets a left half-macron, the last base character gets a right half-macron, and the middle base characters get conjoining macrons; for example, ⟨ϥⲛⲧ⟩ with a macron above is written ⟨ϥ︤ⲛ︦ⲧ︥⟩. These characters were encoded primarily for use in Coptic, but that document also gives a Latin example.

http://www.unicode.org/L2/L2011/11224-n4078.pdf generalizes that system to other diacritics. Each multi-character diacritic is encoded as a left piece above/below the first base, a right piece above/below the last base, and conjoining macrons above/below the middle bases.

http://www.unicode.org/L2/L2013/13139-half-marks.pdf adds U+FE2E and U+FE2F. For example, ⟨црь⟩ with a titlo above is written ⟨ц︮р︦ь︯⟩.

== Font issues ==

As you point out, it is difficult to get these characters to render properly, mainly because the width depends on the base character and the height depends on how many other diacritics the base character has. I don’t think it is possible to do a fully general solution that will work for all inputs. Therefore, the best you can do is something that works for most plausible inputs.

=== Width ===

Let’s say it is impossible to know how wide the base character is. Therefore, these diacritics must have widths appropriate to all bases. That means the left and right half-diacritics should be narrow enough to look good on ⟨i⟩ and the conjoining middle diacritics should be wide enough to cover ⟨w⟩ without going too far over an adjacent narrow character.

=== Height ===

Let’s say it is impossible to know how many other diacritics the base character has or whether the base has a descender or ascender. Therefore, these diacritics must be quite high/low in order to avoid overlapping. For example, a tall base is ⟨Ê⟩ and a deep base is ⟨p⟩.

Or maybe ⟨Ê⟩ is too tall; maybe these diacritics would look better at caps-height, even if that would overlap the circumflex in ⟨Ê⟩. That’s the sort of trade-off a font designer is more capable to answer than me.

=== Mark positioning ===

Don’t bother with mark positioning for these characters. Otherwise, different parts of the multi-part diacritic might be positioned at different heights. For example, in ⟨s︧j︨⟩, if the right half-mark is lowered to avoid touching the descender, the left half-mark won’t be lowered, so the mark will be split into two pieces.

I think it is better to have overlaps than split diacritics. Therefore, I suggest doing the dumb pre-OpenType thing and placing these diacritics at constant heights.

== Conclusion ==

I think my suggestion covers everything plausible without looking too bad. It’s a good place to start anyway. If I find anything that it doesn’t handle, I’ll report it here. I promise not to report bugs for implausible unattested edge cases.

David Corbett <dscorbett>
Mon 20 Mar 2017 09:34:58 AM UTC, comment #2:

Hi David,

so... do you have any specification of how these marks should work, beyond the one I mentioned in the Unicode Book? It talks about Copitc -- but I don't really know the breadth of application of these marks.

For the record, the technical problems I see are these:

1) as far as I know, mark alignment works only on consecutive marks -- if marks are separated by base glyphs, nothing can be expected of them.

2) the left mark can not know how wide its preceding character is, and so no way to know where to place itself, to connect with a previous mark.

3) intermediate marks also have no way to know how wide their character is.

A solution to (3) might be a substitution lookup, that puts a wider line over an m than an i.

That solution is impractical for the right-side marks: it would involve a different case for nearly every letter in every alphabet for which the marks are used.

However, it should be possible to reverse the order of the right-side mark and the last base glyph, so that mark positioning can be applied with the previous mark.

Steve White <Stevan_White>
Project AdministratorIn charge of this item.
Sun 19 Mar 2017 11:19:31 PM UTC, comment #1:

Hi David,

Yes there are several things that could be improved here. Frankly I only tested it with a few combinations of two letters and certain combinations of marks.

But I don't find much in the way of how this is supposed to be encoded. There's only the Unicode Boook 7.2, which gives only the example with two letters, and no clue as to how it should be implemented.

What's worse, I don't know how simple mark positioning can work with the encoding you give, without rearranging the glyphs. (If all the marks were placed directly after the first base character in the sequence, it works easily.) Any ideas?

Steve White <Stevan_White>
Project AdministratorIn charge of this item.
Sun 19 Mar 2017 04:00:09 PM UTC, original submission:

In FreeSerif, the combining half marks in the range U+FE20 to U+FE2F do not join each other properly.

U+FE20, U+FE21, U+FE22, and U+FE23 are too high: they should be at the height of U+FE26.

U+FE20, U+FE21, U+FE22, U+FE23, U+FE2E, and U+FE2F are too thin or too thick: they should have the stroke width of U+FE26. Alternatively, U+FE24 through U+FE26 have the wrong stroke width.

U+FE27, U+FE28, U+FE29, and U+FE2A are at the wrong height: they should be at the height of U+FE2D.

U+FE27, U+FE28, U+FE29, and U+FE2A are too thick: they should have the stroke width of U+FE2D. Alternatively, U+FE2B through U+FE2D are too thin.

U+FE22, U+FE23, U+FE29, U+FE2A, U+FE2E, and U+FE2F connect to the adjacent combining half mark with a sloping stroke. This makes them look bad when connecting to U+FE26 or U+FE2D, which connect horizontally.

The combining right half marks U+FE21, U+FE23, U+FE25, U+FE28, U+FE2A, U+FE2C, and U+FE2F combine with the following character. For example, a tilde above ⟨ng⟩ is ⟨n︢g︣⟩, but FreeSerif encourages the user to write ⟨n︢︣g⟩.

To summarize, the following sequences look wrong.

data:text/html;charset=utf-8,<span style="font: 100px FreeSerif">x&%23xFE20;x&%23xFE26;x&%23xFE21;<br>x&%23xFE22;x&%23xFE26;x&%23xFE23;<br>x&%23xFE24;x&%23xFE26;x&%23xFE25;<br>x&%23xFE27;x&%23xFE2D;x&%23xFE28;<br>x&%23xFE29;x&%23xFE2D;x&%23xFE2A;<br>x&%23xFE2B;x&%23xFE2D;x&%23xFE2C;<br>x&%23xFE2E;x&%23xFE26;x&%23xFE2F;

David Corbett <dscorbett>


(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):

Attached Files
file #40055:  combining-macrons.html added by Stevan_White (4KiB - text/html - with Cyrillic)


Depends on the following items: None found

Items that depend on this one: None found


Carbon-Copy List
  • -unavailable- added by Stevan_White (Posted a comment)
  • -unavailable- added by dscorbett (Submitted the item)

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.


    Please enter the title of George Orwell's famous dystopian book (it's a date):



    Follow 10 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Wed 22 Mar 2017 09:50:55 AM UTCStevan_WhiteAttached File-=>Added combining-macrons.html, #40056
      StatusProceeding=>Fix posted
    Tue 21 Mar 2017 10:31:59 PM UTCStevan_WhiteAttached File-=>Added combining-macrons.html, #40055
    Tue 21 Mar 2017 06:01:39 PM UTCStevan_WhiteAttached File-=>Added combining-macrons.html, #40050
    Tue 21 Mar 2017 06:01:10 PM UTCStevan_WhiteAttached File#40047=>Removed
    Tue 21 Mar 2017 02:35:40 PM UTCStevan_WhiteAttached File-=>Added combining-macrons-notes, #40047
    Mon 20 Mar 2017 09:05:50 PM UTCStevan_WhiteAttached File-=>Added combining-macrons.html, #40042
      StatusNeed info=>Proceeding
    Sun 19 Mar 2017 11:19:31 PM UTCStevan_WhiteStatusNone=>Need info
      Assigned toNone=>Stevan_White

    Back to the top

    Powered by Savane 3.1-cleanup1