helpUnifont - Support: sr #108441, Unifont's "post" Table...

 
 

sr #108441: Unifont's "post" Table Has Too Many Entries for W3C WOFF 1.0 Validator

Submitted by:  Paul Hardy <unifoundry>
Submitted on:  Mon 18 Nov 2013 04:41:56 AM UTC  
 
Category: NonePriority: 5 - Normal
Severity: 1 - WishStatus: In Progress
Privacy: PublicAssigned to: Paul Hardy <unifoundry>
Open/Closed: OpenOperating System: None

Add a New Comment (Rich MarkupRich Markup):
   

You are not logged in

Please log in, so followups can be emailed to you.

 

Sat 15 Feb 2014 11:41:54 PM UTC, comment #4:

The bug on Mozilla is now has a status of "RESOLVED FIXED". So Mozilla has incorporated the change to support a WOFF version of Unifont in Firefox. The W3C's WOFF Validator remains to be resolved.

Paul Hardy <unifoundry>
Project AdministratorIn charge of this item.
Wed 20 Nov 2013 04:14:39 PM UTC, comment #3:

This bug report has just been filed at Mozilla, requesting that they update to the latest version of OTS:

https://bugzilla.mozilla.org/show_bug.cgi?id=941019

Paul Hardy

Paul Hardy <unifoundry>
Project AdministratorIn charge of this item.
Wed 20 Nov 2013 04:31:41 AM UTC, comment #2:

A clarification: the table in question is the "post" font table, which lists glyph names; its index value can only reach 32,767 under strict conformance with the TrueType/OpenType spec. This font table provides information for PostScript printers.

Google has just allowed higher index values in an OpenType "post" table in version r106 of OTS, their "Sanitiser for OpenType" for their Chromium projects:

https://code.google.com/p/ots/source/detail?r=106

https://codereview.chromium.org/76363004/

Thanks to Brad Nelson, "ksakamoto" and "ysukes" for such rapid turnaround!

Jonathan Kew of Firefox has also suggested an alternative of using a TrueType/OpenType version 3.0 post table, which has no glyph names:

http://www.microsoft.com/typography/otspec/post.htm

This has the benefit of making the font file smaller, which seems advantageous once Unifont includes a WOFF version. The link above does, however, caution that: "The printing behavior of this version [3.0] on PostScript printers is unspecified". So using a version 3.0 "post" table might only be something to consider for the WOFF version which, being downloadable over the Web, would benefit most from a smaller file size.

Paul Hardy

Paul Hardy <unifoundry>
Project AdministratorIn charge of this item.
Tue 19 Nov 2013 01:55:46 PM UTC, comment #1:

Google has just patched Chrome to allow fonts with glyph index values > 32,767:

https://codereview.chromium.org/68173026/

Paul Hardy

Paul Hardy <unifoundry>
Project AdministratorIn charge of this item.
Mon 18 Nov 2013 04:41:56 AM UTC, original submission:

Unifont does not have an official WOFF (web font) version, but it would be nice to add to the distribution. The W3C WOFF validator imposes Microsoft's original limitation of glyphs with index values from 0 through 32,767, inclusive. FireFox uses this strict validator, which fails a WOFF version of Unifont as invalid:

https://bugzilla.mozilla.org/show_bug.cgi?id=733978

At present, Google's Chrome browser also rejects a WOFF version of Unifont but hopefully Google will allow a workaround to be available; see:

https://codereview.chromium.org/68173026/

Apple's Safari is reported to be able to use a WOFF version of Unifont just fine.

The heart of this issue dates to Apple and Microsoft's early work developing the TrueType spec. Microsoft's TrueType 2.0 spec, released in 2001, mentions this limit for the glyphNameIndex[] array:

http://www.microsoft.com/typography/otspec/post.htm

The spec says: "Index numbers 32768 through 65535 are reserved for future use." Thus Microsoft restricts the 16-bit index value to the lower 15 bits (2^15 = 32,768; 2^16 = 65,536).

O'Reilly's Fonts & Encodings, first edition by Yannis Haralambous, Appendix D, p. 705 gives some historical perspective. This book mentions that Apple first developed a prototype for TrueType using their 32-bit operating system. Microsoft then tweaked the standard for Microsoft Windows 3.1, which was a 16-bit operating system.

Therefore it appears that Microsoft reserved the high bit of the 16-bit glyph index so that they could safely index a TrueType glyph array using a signed 16-bit integer under Microsoft Windows 3.1.

The standard version of Unifont 6.3 contains over 50,000 glyphs. This TrueType font works fine under Microsoft Windows XP, Windows Vista, Windows 7, MacOS X, as well as GNU/Linux and Unix systems such as Solaris and FreeBSD. The issue is the W3C's WOFF validator.

That validator considers a font invalid if it contains glyphs with index values greater than 32,767, because according to the 2001 Microsoft TrueType 2.0 spec, higher values up to 65,535 are reserved for future use.

In this Unicode day and age, this restriction to a 15-bit index value seems an anachronism. I intend to request that the W3C and others validate the font using multiple return values along the lines of "strictly conforms" (maximum index < 32,768), "loosely conforms" (maximum index up to 65,535), and "invalid" (index value too large to represent in an unsigned 16-bit number).

Apart from Unifont, there is at least one other font and one other Unicode script that would also fail the W3C's strict 15-bit indexing limitation. They would thus benefit from my proposed loose conformance to TrueType.

The other font is Wen Quan Yi, with over 40,000 glyphs in multiple point sizes (Unifont is only available in a single point size). This is by far the best and most popular free font for Chinese ideographs, present on GNU/Linux and many other systems too numerous to mention. The English version of Wen Quan Yi's home page is here:

http://wenq.org/wqy2/index.cgi?action=browse&id=Home&lang=en

The other Unicode script is "CJK Unified Ideographs Extension B" containing over 40,000 ideographs as of the Unicode 6.3 release. See

ftp://ftp.unicode.org/Public/6.3.0/ucd/Unihan.zip

for a compressed ASCII list of all assigned code points in Plane 2 (which must be uncompressed and sorted uniquely on the first value), and

http://www.unicode.org/charts/PDF/U20000.pdf

for the code chart on just the "CJK Unified Ideographs Extension B" script. Warning: the PDF code chart file is 40 Mbytes.

The Unihan.zip file can be used to verify the presence of unique Unicode CJK ideograph code points.

The code chart shows values ranging from U+20000 through U+2A6D6, inclusive. This is a range of 42,711 possible glyphs (ideographs) just in that one script.

Therefore using Microsoft's 2001 limitation of index values not exceeding 32,767, it is not possible to have a single font that contains only the Unicode "CJK Unified Ideographs Extension B" script and nothing more. An implementation of that one Unicode script would have to be split across two fonts.

Stretching the strict interpretation of Microsoft's 2001 TrueType spec to accept a full 16-bit glyph index value alleviates this problem. Hence the proposal that a WOFF validator return at least three values: "strictly conforms", "loosely conforms", and "invalid" or some variation on that theme.

This suggestion does not propose attempting to exceed the 16-bit TrueType glyph index limit; that is a hard limit of the TrueType spec. Any exception to that would require a newer extended font format.

Paul Hardy

Paul Hardy <unifoundry>
Project AdministratorIn charge of this item.

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by unifoundry (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 2 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Wed 20 Nov 2013 04:32:50 AM UTCunifoundrySummaryUnifont Has Too Many Glyphs for W3C WOFF 1.0 Validator=>Unifont's "post" Table Has Too Many Entries for W3C WOFF 1.0 Validator
    Mon 18 Nov 2013 04:42:50 AM UTCunifoundryAssigned toNone=>unifoundry

    Back to the top


    Powered by Savane 3.1-cleanup