/[freetype]/freetype2/src/gxvalid/README
ViewVC logotype

Diff of /freetype2/src/gxvalid/README

Parent Directory Parent Directory | Revision Log Revision Log | View Patch Patch

revision 1.1 by mpsuzuki, Wed Aug 24 04:31:31 2005 UTC revision 1.2 by wl, Wed Aug 31 07:13:27 2005 UTC
# Line 1  Line 1 
1    gxvalid: TrueType GX validator  gxvalid: TrueType GX validator
2    ==============================  ==============================
3    
4    1. What is this  
5    ---------------  1. What is this
6    "gxvalid" is a module to validate TrueType GX tables:  a collection of  ---------------
7    additional  tables  in  TrueType font which is used  by  "QuickDraw GX  
8    Text",  Apple  Advanced Typography  (AAT).  In addition,  gxvalid  can    `gxvalid' is a module to  validate TrueType GX tables: a collection of
9    validates "kern" table which had been extended for AAT. Like otvalid,    additional tables  in TrueType  font which are  used by  `QuickDraw GX
10    gxvalid uses Freetype2's validator framework(ftvalid).    Text',  Apple Advanced  Typography  (AAT).  In  addition, gxvalid  can
11      validates `kern'  tables which have  been extended for AAT.   Like the
12      otvalid  module,   gxvalid  uses  Freetype   2's  validator  framework
13      (ftvalid).
14    
15    You can link gxvalid with your program; before running your own layout    You can link gxvalid with your program; before running your own layout
16    engine,  gxvalid validates a font file.  As the result, you can reduce    engine, gxvalid validates a font  file.  As the result, you can remove
17    error-checking code  from the layout engine.  You can use gxvalid as a    error-checking code  from the layout  engine.  It is also  possible to
18    stand-alone font validator;  ftvalid command included in ft2demo calls    use  gxvalid  as a  stand-alone  font  validator;  the `ftvalid'  test
19    gxvalid internally.  Stand-alone font validator may be useful for font    program  included  in the  ft2demo  bundle  calls gxvalid  internally.
20    developers.    A stand-alone font validator may be useful for font developers.
21    
22      This documents documents the following issues.
23    
   This documents contains following informations:  
24    - supported TrueType GX tables    - supported TrueType GX tables
25    - validation limitation in principle    - fundamental validation limitations
26    - permissive error handling of broken GX tables    - permissive error handling of broken GX tables
27    - "kern" table issue.    - `kern' table issue.
28    
29    
30    2. Supported tables
31    -------------------
32    
33      The following GX tables are currently supported.
34    
35        bsln
36        feat
37        just
38        kern(*)
39        lcar
40        mort
41        morx
42        opbd
43        prop
44        trak
45    
46    2. Supported tables    The following GX tables are currently unsupported.
   -------------------  
   Following GX tables are currently supported.  
     bsln feat just kern(*) lcar mort morx opbd prop trak  
47    
48    Following GX tables are currently unsupported.      cvar
49      cvar fdsc fmtx fvar gvar Zapf      fdsc
50        fmtx
51        fvar
52        gvar
53        Zapf
54    
55    Following GX tables won't be supported.    The following GX tables won't be supported.
     acnt(**) hsty(***)  
56    
57    Undocumented tables in TrueType fonts designed for Apple platform.      acnt(**)
58      CVTM TPNM addg umif      hsty(***)
59    
60    *)   "kern" validator includes both of classic kern  (format supported    The following undocumented tables in TrueType fonts designed for Apple
61         by both of Microsoft and Apple platforms) and new kern  (a format    platform aren't handled either.
        supported by Apple platform only).  
62    
63    **)  "acnt" tables is not supported  by currently available Apple font      addg
64        CVTM
65        TPNM
66        umif
67    
68    
69      *)   The `kern'  validator handles both  the classic and the  new kern
70           formats;  the former  is supported  on both  Microsoft  and Apple
71           platforms, while the latter is supported on Apple platforms.
72    
73      **)  `acnt' tables are not supported by currently available Apple font
74         tools.         tools.
75    
76    ***) There is one more Apple extension "hsty" but it is for Newton-OS,    ***) There  is  one more  Apple  extension,  `hsty',  but  it  is  for
77         not GX  (Newton-OS  is a platform by Apple,  but it can use sfnt-         Newton-OS, not GX  (Newton-OS is a platform by  Apple, but it can
78         housed bitmap fonts only.   Therefore, it should be excluded from         use  sfnt- housed bitmap  fonts only).   Therefore, it  should be
79         "Apple platform" in the context of TrueType.   gxvalid ignores it         excluded  from  `Apple  platform'  in the  context  of  TrueType.
80         as Apple font tools do so.         gxvalid ignores it as Apple font tools do so.
81    
82    We have checked 183 fonts bundled to MacOS 9.1, MacOS 9.2, MacOS 10.0,  
83    MacOS X 10.1, MSIE for MacOS and AppleWorks 6.0.  In addition, we have    We have  checked 183  fonts bundled with  MacOS 9.1, MacOS  9.2, MacOS
84    checked  67 Dynalab fonts  (designed  for  MacOS) and  189 Ricoh fonts    10.0, MacOS X 10.1, MSIE  for MacOS, and AppleWorks 6.0.  In addition,
85    (designed for  Windows and MacOS dual platforms).  The number of fonts    we have  checked 67 Dynalab fonts  (designed for MacOS)  and 189 Ricoh
86    including TrueType GX tables are listed in following:    fonts (designed for Windows and  MacOS dual platforms).  The number of
87      fonts including TrueType GX tables are as follows.
88    
89      bsln:  76      bsln:  76
90      feat: 191      feat: 191
91      just:  84      just:  84
# Line 65  Line 96 
96      opbd:   4      opbd:   4
97      prop: 114      prop: 114
98      trak:  16      trak:  16
   Dynalab and Ricoh fonts didn't have GX tables except of feat and mort.  
99    
100    3. Validation limitations in principle    Dynalab  and Ricoh fonts  don't have  GX tables  except of  `feat' and
101    --------------------------------------    `mort'.
102    TrueType GX provides  layout information to font-rasterize/text-layout  
103    libraries.  gxvalid can check whether layout information is  stored as  
104    TrueType GX  format specified by  Apple.  But gxvalid cannot check how  3. Fundamental validation limitations
105    QuickDraw GX/AAT renderer uses the stored information.  -------------------------------------
106    
107      3-1. Validation of State Machine activity    TrueType  GX  provides  layout   information  to  libraries  for  font
108      -----------------------------------------    rasterizers  and text layout.   gxvalid can  check whether  the layout
109      QuickDraw GX/AAT  has  "State Machine"  to provide "stateful" layout    data in  a font is conformant  to the TrueType GX  format specified by
110      features,  and TrueType GX  stores  the  state transition diagram of    Apple.  But gxvalid cannot check  a how QuickDraw GX/AAT renderer uses
111      "State Machine" in "StateTable" data structure.  While State Machine    the stored information.
112      receives  a series of glyph ID,  State Machine starts from "start of  
113      text" state,  walks  around  various  states and  generates  various    3-1. Validation of State Machine activity
114      layout  informations  to renderer,  and  finally  reaches to "end of    -----------------------------------------
115      text".  
116        QuickDraw GX/AAT uses a `State Machine' to provide `stateful' layout
117        features,  and TrueType GX  stores the  state transition  diagram of
118        this `State  Machine' in a  `StateTable' data structure.   While the
119        State  Machine receives  a series  of glyph  IDs, the  State Machine
120        starts with `start  of text' state, walks around  various states and
121        generates various  layout informations to the  renderer, and finally
122        reaches the `end of text' state.
123    
124      gxvalid can check essential errors like:      gxvalid can check essential errors like:
125        - possibility of state transition to undefined states  
126        - existence  of  glyph ID that State Machine  doesn't know  how to        - possibility of state transitions to undefined states
127          handle it        - existence of glyph  IDs that the State Machine  doesn't know how
128        - State Machine cannot  compute the layout information  from given          to handle
129          diagram        - the  State Machine  cannot compute  the layout  information from
130      these errors  can be checked within finite steps,  and without State          given diagram
131      Machine itself,  because  these are  errors of "expression" of state  
132        These errors  can be  checked within finite  steps, and  without the
133        State Machine itself, because these are `expression' errors of state
134      transition diagram.      transition diagram.
135    
136      There's no limitation about how long State Machine walks around,  so      There  is no  limitation  about  how long  the  State Machine  walks
137      validation of the algorithm in the state transition diagram requires      around,  so validation  of  the algorithm  in  the state  transition
138      infinite steps, even if we have State Machine in gxvalid. Therefore,      diagram requires infinite  steps, even if we had  a State Machine in
139      following "errors" cannot be checked.      gxvalid.   Therefore, the  following errors  and problems  cannot be
140        - existence of states which State Machine never transits to.      checked.
141        - possibility that State Machine never reaches to "end of text".  
142        - possibility of stack underflow/overflow in State Machine        - existence of states which the State Machine never transits to
143          (in  ligature  and  contextual glyph substitution, State Machine        - the  possibility that the  State Machine  never reaches  `end of
144           can store 16 glyphs onto its stack)          text'
145          - the possibility of stack underflow/overflow in the State Machine
146      In addition,  gxvalid doesn't check  "temporal glyph ID" used in the          (in  ligature  and  contextual  glyph substitutions,  the  State
147      chained State Machines (in "mort" and "morx" tables).  When a layout          Machine can store 16 glyphs onto its stack)
148      feature is implemented by  single State Machine,  glyph ID converted  
149      by State Machine is passed to the glyph renderer, thus it should not      In addition, gxvalid doesn't check `temporary glyph IDs' used in the
150      point to undefined glyph ID.  But if a layout feature is implemented      chained State Machines  (in `mort' and `morx' tables).   If a layout
151      by chained State Machines, the component State Machine (if it is not      feature  is  implemented by  a  single  State  Machine, a  glyph  ID
152      final one) is permitted to generate  undefined glyph ID for temporal      converted by the State Machine is passed to the glyph renderer, thus
153      use, because it is handled  by next component State Machine, instead      it  should not  point to  an undefined  glyph ID.   But if  a layout
154      of the glyph renderer.  To validate such temporal glyph ID,  gxvalid      feature is implemented by  chained State Machines, a component State
155      must stack  all undefined glyph IDs  which is possible in the output      Machine  (if it  is  not the  final  one) is  permitted to  generate
156      of previous State Machine and search them in "ClassTable" of current      undefined glyph IDs for temporary use, because it is handled by next
157      State Machine.  It is too complexed work to  list all possible glyph      component State Machine and not  by the glyph renderer.  To validate
158      IDs from StateTable, especially from ligature substitution table.      such temporary glyph IDs, gxvalid must stack all undefined glyph IDs
159        which  can occur in  the output  of the  previous State  Machine and
160      3-2. Validation of relationship among multiple layout features      search  them in  the  `ClassTable' structure  of  the current  State
161      --------------------------------------------------------------      Machine.  It is too complex to  list all possible glyph IDs from the
162      gxvalid  does not  validate  the relationship  among multiple layout      StateTable, especially from a ligature substitution table.
163    
164      3-2. Validation of relationship between multiple layout features
165      ----------------------------------------------------------------
166    
167        gxvalid does  not validate the relationship  between multiple layout
168      features at all.      features at all.
169    
170      If  multiple layout features are defined in  TrueType GX tables, the      If  multiple layout  features  are defined  in  TrueType GX  tables,
171      interactivity,  overriding,  and conflict  among layout features are      possible  interactions,  overrides,  and  conflicts  between  layout
172      defined in the font too.  For example,  there are several predefined      features are implicitly  given in the font too.   For example, there
173      spacing control features:      are several predefined spacing control features:
174    
175        - Text Spacing          (Proportional/Monospace/Half-width/Normal)        - Text Spacing          (Proportional/Monospace/Half-width/Normal)
176        - Number Spacing        (Monospaced-numbers/Proportional-numbers)        - Number Spacing        (Monospaced-numbers/Proportional-numbers)
177        - Kana Spacing          (Full-width/Proportional)        - Kana Spacing          (Full-width/Proportional)
178        - Ideographic Spacing   (Full-width/Proportional)        - Ideographic Spacing   (Full-width/Proportional)
179        - CJK Roman Spacing     (Half-width/Proportional/Default-roman        - CJK Roman Spacing     (Half-width/Proportional/Default-roman
180                                 /Full-width-roman/Proportional)                                 /Full-width-roman/Proportional)
181      If  all layout features  are  independently  managed,  we can set an  
182      inconsistent  typographic rule, as like "Text Spacing=Monospace" and      If all  layout features are  independently managed, we  can activate
183      "Ideographic Spacing=Proportional", at the same time.      inconsistent  typographic rules  like  `Text Spacing=Monospace'  and
184        `Ideographic Spacing=Proportional' at the same time.
185      The combination  of each layout feature  is managed by 32bit integer  
186      (1 bit for 1 selector setting),  so we can define relationship among      The combinations  of layout features  is managed by a  32bit integer
187      features  up  to  32 settings,  theoretically.  But  if  setting  of      (one bit each for selector  setting), so we can define relationships
188      a feature affects setting of another features,  typographic priority      between  up  to 32  features,  theoretically.   But  if one  feature
189      of  each  layout  feature  is required to validate the relationship.      setting  affects  another   feature  setting,  we  need  typographic
190        priority  rules to  validate the  relationship.   Unfortunately, the
191      TrueType GX format specification does not give such information even      TrueType GX format specification does not give such information even
192      for predefined features.      for predefined features.
193    
   4. Permissive error handling of broken GX tables  
   ------------------------------------------------  
   When Apple's font rendering system finds  an inconsistency,  violation  
   of specification  or unspecified value in TrueType GX tables,  they do  
   not always return error. In most case, they silently ignore such wrong  
   values or  whole  of  table.  In fact,  MacOS  is  shipped  with fonts  
   including  broken  GX/AAT  tables,  but  no  harmful  effects  due  to  
   officially broken fonts are observed by end-users.  
   
   gxvalid  is designed to continue its validation  as long as possible.  
   When gxvalid find wrong value,  gxvalid warns it at least,  and take a  
   fallback procedure if possible.  The fallback procedure depends on the  
   debug level.  
194    
195    We used following 3 tools to refer Apple's error handling.  4. Permissive error handling of broken GX tables
196    ------------------------------------------------
197    
198      When  Apple's font  rendering system  finds an  inconsistency,  like a
199      specification  violation or  an  unspecified value  in  a TrueType  GX
200      table, it does not always  return error.  In most cases, the rendering
201      engine silently  ignores such wrong  values or even whole  tables.  In
202      fact, MacOS is shipped with  fonts including broken GX/AAT tables, but
203      no harmful  effects due to  `officially broken' fonts are  observed by
204      end-users.
205    
206      gxvalid  is designed  to continue  the validation  process as  long as
207      possible.  When gxvalid find wrong  values, gxvalid warns it at least,
208      and takes  a fallback procedure  if possible.  The  fallback procedure
209      depends on the debug level.
210    
211      We used the following three tools to investigate Apple's error handling.
212    
213      - FontValidator  (for MacOS 8.5 - 9.2)  resource fork font      - FontValidator  (for MacOS 8.5 - 9.2)  resource fork font
214      - ftxvalidator   (for MacOS X 10.1 -)   dfont or naked-sfnt      - ftxvalidator   (for MacOS X 10.1 -)   dfont or naked-sfnt
215      - ftxdumperfuser (for MacOS X 10.1 -)   dfont or naked-sfnt      - ftxdumperfuser (for MacOS X 10.1 -)   dfont or naked-sfnt
   However,  all tests are on PowerPC based Macintosh, we have not tested  
   on m68k-based Macintosh at all, at present.  
216    
217    We  checked  183 fonts  bundled to  MacOS 9.1,  MacOS 9.2, MacOS 10.0,    However, all tests were done on a PowerPC based Macintosh; at present,
218    MacOS X 10.1,  MSIE for MacOS  and  AppleWorks 6.0.  These  fonts  are    we have not checked those tools on a m68k-based Macintosh.
   distributed  officially,  but many  broken GX/AAT tables  are found by  
   Apple's font tools. In following, we list typical violation against GX  
   specification, in Apple official fonts.  At least, gxvalid warns them,  
   and fallback method to continue  
219    
220      4-1. broken BinSrchHeader ( 19/183)    In total, we checked 183 fonts  bundled to MacOS 9.1, MacOS 9.2, MacOS
221      -----------------------------------    10.0, MacOS X  10.1, MSIE for MacOS, and  AppleWorks 6.0.  These fonts
222      BinSrchHeader is a header of data array, for m68k platform to access    are distributed  officially, but many broken GX/AAT  tables were found
223      memory effectively. Although independent parameters for real use are    by Apple's font tools.  In the following, we list typical violation of
224      only  2  (unitSize  and  nUnits),  BinSrchHeader  has  3  additional    the GX specification, in fonts officially distributed with those Apple
225      parameters which can be  calculated  from  unitSize and  nUnits, for    systems.
226      fast setup.  Apple font tools ignore them silently, so gxvalid warns  
227      inconsistency  and  always  continues  validation.   The  additional    4-1. broken BinSrchHeader (19/183)
228      parameters are ignored regardless of the consistency.    ----------------------------------
229    
230        `BinSrchHeader' is  a header of a  data array for  m68k platforms to
231        access memory efficiently.  Although  there are only two independent
232        parameters  for real  (`unitSize' and  `nUnits'),  BinSrchHeader has
233        three additional parameters which  can be calculated from `unitSize'
234        and  `nUnits',  for  fast  setup.   Apple  font  tools  ignore  them
235        silently, so gxvalid warns if it finds and inconsistency, and always
236        continues  validation.    The  additional  parameters   are  ignored
237        regardless of the consistency.
238    
239          19  fonts include  such  inconsistencies; all  breaks  are in  the
240          BinSrchHeader structure of the `kern' table.
241    
242      4-2. too-short LookupTable (5/183)
243      ----------------------------------
244    
245        LookupTable format 0  is a simple array to get a  value from a given
246        GID (glyph  ID); the index of  this array is a  GID too.  Therefore,
247        the length  of the array is expected  to be same as  the maximum GID
248        value defined  in the `maxp' table,  but there are  some fonts whose
249        LookupTable format 0 is too  short to cover all GIDs.  FontValidator
250        ignores  this error silently,  ftxvalidator and  ftxdumperfuser both
251        warn and continue.  Similar problems are found in format 3 subtables
252        of `kern'.  gxvalid  warns always and abort if  the validation level
253        is set to FT_VALIDATE_PARANOID.
254    
255          5 fonts include too-short kern format 0 subtables.
256          1 font includes too-short kern format 3 subtable.
257    
258      4-3. broken LookupTable format 2 (1/183)
259      ----------------------------------------
260    
261        LookupTable  format  2,  subformat  4  covers the  GID  space  by  a
262        collection  of  segments which  are  specified  by `firstGlyph'  and
263        `lastGlyph'.   Some  fonts  store  `firstGlyph' and  `lastGlyph'  in
264        reverse order,  so the segment specification is  broken.  Apple font
265        tools ignore this error silently;  a broken segment is ignored as if
266        it  did not  exist.   gxvalid  warns and  normalize  the segment  at
267        FT_VALIDATE_DEFAULT, or ignore  the segment at FT_VALIDATE_TIGHT, or
268        abort at FT_VALIDATE_PARANOID.
269    
270      19 fonts include  inconsistent with calculated values        1 font includes broken LookupTable format 2, in the `just' table.
        all breaks are in BinSrchHeader of "kern" table.  
271    
272      4-2. too-short LookupTable ( 5/183)      *) It seems  that all fonts manufactured by  ITC for AppleWorks have
273      -----------------------------------         this error.
     LookupTable  format 0 is simple array to get a value from given GID,  
     the  index  of  array  is  GID.  Therefore,  the  length of array is  
     expected to be same with max GID defined in "maxp" table,  but there  
     is  some fonts  whose LookupTable format 0 is too short to cover all  
     GID.  FontValidator ignores  this error  silently,  ftxvalidator and  
     ftxdumperfuser  warns  and continues.  Similar  shortage is found in  
     format 3 subtable of "kern".  
     gxvalid warns always and abort at FT_VALIDATE_PARANOID.  
   
       5 fonts include  too-short kern format 0 subtables.  
       1 font  includes too-short kern format 3 subtable.  
   
   
     4-3. broken LookupTable format 2 ( 1/183)  
     -----------------------------------------  
     LookupTable format 2, 4 covers GID  space  by collection of segments  
     which  specified  by  firstGlyph  and  lastGlyph.  Some fonts stores  
     firstGlyph and lastGlyph in reverse order,  so segment specification  
     is broken.  Apple  font tools ignores this  error  silently,  broken  
     segment is  ignored  as if it  did  not  exist.  gxvalid  warns  and  
     normalize the segment at FT_VALIDATE_DEFAULT,  or ignore the segment  
     at FT_VALIDATE_TIGHT, or abort at FT_VALIDATE_PARANOID.  
274    
275        1 font includes  broken LookupTable format 2, in "just" table.    4-4. bad bracketing in glyph property (14/183)
276      ----------------------------------------------
277    
278      *) It seems that  all fonts manufactured by  ITC for AppleWorks have      GX/AAT defines a  `bracketing' property of the glyphs  in the `prop'
279         this error.      table,  to control layout  features of  strings enclosed  inside and
280        outside  of   brackets.   Some  fonts   give  inappropriate  bracket
281        properties  to glyphs.   Apple  font tools  warn  about this  error;
282        gxvalid warns too and aborts at FT_VALIDATE_PARANOID.
283    
284          14 fonts include wrong bracket properties.
285    
     4-4. bad bracketing in glyph property ( 14/183)  
     -----------------------------------------------  
     GX/AAT defines bracketing property of the glyphs by "prop" table, to  
     control layout functionalities for string closed in brackets and out  
     of brackets.  Some fonts  give  inappropriate bracket properties  to  
     glyphs.  Apple font tools warn this error.  gxvalid warns always and  
     abort at FT_VALIDATE_PARANOID.  
286    
287       14 fonts include  wrong bracket properties.    4-5. invalid feature number (117/183)
288      -------------------------------------
289    
290        The GX/AAT extension can  include 255 different layout features, but
291        popular      layout      features      are      predefined      (see
292        http://developer.apple.com/fonts/Registry/index.html).   Some  fonts
293        include feature  numbers which are incompatible  with the predefined
294        feature registry.
295    
296        In our survey, there are 140 fonts including `feat' table.
297    
298      4-5. invalid feature number (117/183)      a) 67 fonts use a feature number which should not be used.
299      -------------------------------------      b) 117 fonts set the wrong feature range (nSetting).  This is mostly
300      GX/AAT extension can include 255 different features for layout,  but         found in the `mort' and `morx' tables.
     popular layout features are predefined  
     (see http://developer.apple.com/fonts/Registry/index.html).  
     Some  fonts  include  feature  number  which  is  incompatible  with  
     predefined feature registry.  
   
     In our survey, there are 140 fonts including "feat" table.  
     a)  67 fonts uses feature number which should not be used.  
     b) 117 fonts set wrong feature range (nSetting).  
            this infraction is found in mort/morx.  
301    
302      Apple font tools gives no warning,  although  they  cannot recognize      Apple  font tools give  no warning,  although they  cannot recognize
303      what  the  feature  is.  At  FT_VALIDATE_DEFAULT,  gxvalid warns but      what  the feature  is.   At FT_VALIDATE_DEFAULT,  gxvalid warns  but
304      continues in both cases (a, b).  At FT_VALIDATE_TIGHT, gxvalid warns      continues in both cases (a, b).  At FT_VALIDATE_TIGHT, gxvalid warns
305      and aborts for (a), but continues for (b).  At FT_VALIDATE_PARANOID,      and aborts for (a), but continues for (b).  At FT_VALIDATE_PARANOID,
306      gxvalid warns and aborts in both cases (a, b).      gxvalid warns and aborts in both cases (a, b).
307    
308      4-6. invalid prop version ( 10/183)    4-6. invalid prop version (10/183)
309      -----------------------------------    ----------------------------------
310      As  most  TrueType  GX  tables,  prop  table  must  start with 32bit  
311      version: 0x00010000, 0x00020000 or 0x00030000.  But some fonts store      As most TrueType GX tables, the `prop' table must start with a 32bit
312      nonsense binary data  in it.  When Apple font tools find them,  they      version identifier: 0x00010000,  0x00020000 or 0x00030000.  But some
313      abort  the processing at once,  and following  data  are  unhandled.      fonts  store nonsense binary  data instead.   When Apple  font tools
314      gxvalid does same always.      find them, they abort the processing immediately, and the data which
315        follows is unhandled.  gxvalid does the same.
316        10 fonts include  broken prop version.  
317          10 fonts include broken `prop' version.
318      All  of  these  fonts  are  classic  TrueType  for  Japanese script,  
319      manufactured by Apple.      All  of these  fonts are  classic  TrueType fonts  for the  Japanese
320        script, manufactured by Apple.
321      4-7. unknown resource name (  2/183)  
322      ------------------------------------    4-7. unknown resource name (2/183)
323      NOTE: THIS IS NOT TRUETYPE GX ERROR    ------------------------------------
324      When  TrueType font  is  stored  in  resource  fork or dfont format,  
325      the data must be tagged as "sfnt" in resource fork index,  to invoke      NOTE: THIS IS NOT A TRUETYPE GX ERROR.
326      TrueType font handler  for the data.  But  the TrueType font data in  
327      "Keyboard.dfont" is tagged as "kbd", and that in  "LastResort.dfont"      If  a TrueType  font is  stored  in the  resource fork  or in  dfont
328      is  tagged  as "lst".  Apple  font  tools  can detect the data is of      format, the data must be tagged as `sfnt' in the resource fork index
329      TrueType and successfully validate them.  Possibly this because they      to invoke TrueType font handler for the data.  But the TrueType font
330      are known  to be dfont.  Current  implementation  of  resource  fork      data  in   `Keyboard.dfont'  is  tagged   as  `kbd',  and   that  in
331      driver of FreeType cannot do that, thus gxvalid cannot validate them.      `LastResort.dfont' is tagged as  `lst'.  Apple font tools can detect
332        that the data is in  TrueType format and successfully validate them.
333         2 fonts use unknown tag for TrueType font resource.      Maybe  this is possible  because they  are known  to be  dfont.  The
334        current  implementation  of the  resource  fork  driver of  FreeType
335    5. "kern" table issue      cannot do that, thus gxvalid cannot validate them.
336    ---------------------  
337    In common terminology of TrueType,  "kern"  is classified to basic and        2 fonts use an unknown tag for the TrueType font resource.
338    platform-independent  table.  But there are  Apple extensions of kern,  
339    and  there  is  an  extension  which  requires  GX  state  machine for  5. `kern' table issues
340    contextual kerning.  Therefore,  gxvalid includes  validator for kern.  ----------------------
341    Unfortunately, there is no exact algorithm to check Apple's extension,  
342    so gxvalid  includes pragmatic detector of  data format and  validator    In common terminology of TrueType, `kern' is classified as a basic and
343    for  all possible  data formats,  including data format for Microsoft.    platform-independent table.  But there are Apple extensions of `kern',
344    By calling  classic_kern_validate() instead of gxv_validate(), you can    and  there is  an  extension which  requires  a GX  state machine  for
345    specify available "kern" format explicitly. However, current FreeType2    contextual kerning.   Therefore, gxvalid includes  a special validator
346    uses Microsoft "kern" format only, others are ignored.    for  `kern' tables.   Unfortunately, there  is no  exact  algorithm to
347      check Apple's extension, so  gxvalid includes a heuristic algorithm to
348      5-1. History    find  the proper validation  routines for  all possible  data formats,
349      ------------    including    the   data    format   for    Microsoft.     By   calling
350      Original 16bit version of "kern"  had been designed by Apple in pre-    classic_kern_validate() instead of gxv_validate(), you can specify the
351      GX era, and it was also approved by Microsoft. Afterwards, Apple has    `kern' format  explicitly.  However, current  FreeType2 uses Microsoft
352      designed new 32bit version "kern". Apple has noted as the difference    `kern' format  only, others  are ignored (and  should be handled  in a
353      between  16bit and  32bit version  is only  the size of variables in    library one level higher than FreeType).
354      "kern" header.  In following,  we call the original 16bit version as  
355      "classic", and 32bit version as "new".    5-1. History
356      ------------
357      5-2. Versions and dialects which should be discriminated  
358      --------------------------------------------------------      The original  16bit version of `kern'  was designed by  Apple in the
359      The "kern" table consists of the table header and several subtables.      pre-GX  era, and  it was  also approved  by  Microsoft.  Afterwards,
360      The version  "classic" or  "new" is explicitly written in  the table      Apple designed a  new 32bit version of the  `kern' table.  According
361      header, but there are undocumented difference of font parser between      to  the documentation, the  difference between  the 16bit  and 32bit
362      Microsoft and Apple. It is called as "dialect" in following.      version is only the size of  variables in the `kern' header.  In the
363      There are 3 cases which should be discriminated:  new Apple-dialect,      following,  we call  the original  16bit version  as  `classic', and
364      classic Apple-dialect,  and classic Microsoft-dialect.  Analysis and      32bit version as `new'.
365      auto detection algorithm of gxvalid is described in following.  
366      5-2. Versions and dialects which should be differentiated
367        5-2-1. Version detection: classic and new kern    ---------------------------------------------------------
368        ----------------------------------------------  
369        According   to   Apple  TrueType  specification,   the   clarified      The `kern' table  consists of a table header  and several subtables.
370        difference between classic and new version are only 2:      The version number  which identifies a `classic' or  a `new' version
371          - "kern" table header starts with the version number.      is  explicitly   written  in  the   table  header,  but   there  are
372        undocumented  differences between  Microsoft's and  Apple's formats.
373        It is  called a `dialect' in  the following.  There  are three cases
374        which  should  be  handled:   the  new  Apple-dialect,  the  classic
375        Apple-dialect,  and the classic  Microsoft-dialect.  An  analysis of
376        the formats and the auto detection algorithm of gxvalid is described
377        in the following.
378    
379        5-2-1. Version detection: classic and new kern
380        ----------------------------------------------
381    
382          According  to Apple  TrueType  specification, there  are only  two
383          differences between the classic and the new:
384    
385            - The `kern' table header starts with the version number.
386            The classic version starts with 0x0000 (16bit),            The classic version starts with 0x0000 (16bit),
387            the new version starts with 0x00010000 (32bit).            the new version starts with 0x00010000 (32bit).
388          - In the "kern" table header, the number of subtables follows to  
389            - In the  `kern' table header,  the number of  subtables follows
390            the version number.            the version number.
391            In the classic version, it is stored in 16bit variable.            In the classic version, it is stored as a 16bit value.
392            In the new version, it is stored in 32bit variable.            In the new version, it is stored as a 32bit value.
393    
394        From Apple font tool's output (DumpKERN is also tested in addition        From Apple font tool's output (DumpKERN is also tested in addition
395        to 3 Apple font tools  in above),  there  is  another undocumented        to  the  three  Apple  font  tools in  above),  there  is  another
396        difference.   In new version, the subtable header includes a 16bit        undocumented difference.  In the  new version, the subtable header
397        variable named  "tupleIndex"  which does not exist  in the classic        includes a 16bit variable  named `tupleIndex' which does not exist
398        version.        in the classic version.
399    
400        New version can store all subtable formats (0,  1,  2 and  3), but        The new version  can store all subtable formats (0,  1, 2, and 3),
401        Apple  TrueType  specification  does not  mention  about  subtable        but the Apple TrueType specification does not mention the subtable
402        formats available in classic version.        formats available in the classic version.
403    
404        5-2-2. Avaibale subtable formats in classic version
405        5-2-2. Avaibale subtable format in classic version      ---------------------------------------------------
406        --------------------------------------------------  
407        Although  Apple  TrueType  specification recommends to use classic        Although the  Apple TrueType  specification recommends to  use the
408        version  in the case if the font is designed for both of Apple and        classic version in  the case if the font is  designed for both the
409        Microsoft platforms, it does not note about the available subtable        Apple and Microsoft platforms,  it does not document the available
410        formats in classic version.        subtable formats in the classic version.
411    
412        According to Microsoft TrueType specification, the subtable format        According  to the Microsoft  TrueType specification,  the subtable
413        assured for Windows & OS/2 support is only subtable format 0. Also        format  assured for  Windows  and OS/2  support  is only  subtable
414        Microsoft TrueType specification  describes the subtable format 2,        format  0.  The  Microsoft TrueType  specification  also describes
415        but  does not  mention  about  which  platforms support it.  About        subtable format  2, but does  not mention which  platforms support
416        subtable format 1,  3  and later  are noted as reserved for future        it.  Aubtable formats 1, 3,  and higher are documented as reserved
417        use.  Therefore,  the classic version can store subtable formats 0        for future use.  Therefore, the classic version can store subtable
418        and 2, at least.  ttfdump.exe,  a font tool provided  by Microsoft        formats 0 and 2, at least.  `ttfdump.exe', a font tool provided by
419        ignores  the subtable format  written in the subtable header,  and        Microsoft,  ignores the  subtable format  written in  the subtable
420        parse as if all subtables are in format 0.        header, and parses the table as if all subtables are in format 0.
421    
422        kern subtable format 1 uses StateTable,  so  it cannot be utilized        `kern'  subtable format  1  uses  a StateTable,  so  it cannot  be
423        without  GX State Machine.  Therefore,  it is reasonable to assume        utilized without a GX  State Machine.  Therefore, it is reasonable
424        format 1 (and 3) is introduced after  Apple have introduced GX and        to assume  that format 1 (and  3) were introduced  after Apple had
425        moved to new 32bit version.        introduced GX and moved to the new 32bit version.
426    
427        5-2-3. Apple and Microsoft dialects      5-2-3. Apple and Microsoft dialects
428        -----------------------------------      -----------------------------------
429        The  kern  subtable  has  16bit  "coverage"  to  describe  kerning  
430        attributions,  but bit-interpretations by  Apple and Microsoft are        The  `kern' subtable  has  a 16bit  `coverage'  field to  describe
431        reverse ordered:        kerning attributes, but bit interpretations by Apple and Microsoft
432        e.g. Apple-dialect writes subtable format from 0x000F bit range,        are different:  For example, Apple  uses bits 0-7 to  identify the
433         Microsoft-dialect writes subtable format from 0x0F00 bit range).        subtable, while Microsoft uses bits 8-15.
434    
435        In  addition,  from  the  outputs  of  DumpKERN and FontValidator,        In  addition, due  to the  output of  DumpKERN  and FontValidator,
436        Apple's bit-interpretations of coverage in classic and new version        Apple's bit interpretations of coverage in classic and new version
437        are incompatible. In summary, there are 3 dialects: classic Apple-        are  incompatible also.   In  summary, there  are three  dialects:
438        dialect, classic Microsoft-dialect, and new Apple-dialect.        classic Apple  dialect, classic  Microsoft dialect, and  new Apple
439        The classic Microsoft-dialect and new Apple-dialect are documented        dialect.  The classic Microsoft  dialect and the new Apple dialect
440        by each vendors' TrueType font specification, but the document for        are documented  by each vendors' TrueType  font specification, but
441        classic Apple-dialect had been lost.        the documentation for classic Apple dialect is not available.
442                
443        For example, in new Apple-dialect, the bit 0x8000 is documented as        For example,  in the  new Apple dialect,  bit 15 is  documented as
444        "set to 1 when  the kerning is  vertical".  On the other hand,  in        `set to  1 if  the kerning  is vertical'.  On  the other  hand, in
445        classic Microsoft-dialect, the bit 0x0001 is documented as "set to        classic Microsoft dialect, bit 1 is documented as `set to 1 if the
446        1 when the kerning is  horizontal".  From  the outputs of DumpKERN        kerning  is  horizontal'.   From   the  outputs  of  DumpKERN  and
447        and FontValidator, classic Apple-dialect recognizes the bit 0x8000        FontValidator, classic  Apple dialect recognizes  15 as `set  to 1
448        as "set to 1 when the kerning is horizontal".  From the results of        when  the kerning  is horizontal'.   From the  results  of similar
449        similar experiments, classic Apple-dialect is ein ndian-reverse of        experiments, classic Apple dialect  seems to be the Endian reverse
450        classic Microsoft-dialect.        of the classic Microsoft dialect.
451    
452        It must be noted:  no font tool can sense classic Apple-dialect or        As a  conclusion it must be  noted that no font  tool can identify
453        classic-Microsoft dialect automatically.        classic Apple dialect or classic Microsoft dialect automatically.
454    
455        5-2-4. gxvalid auto dialect detection algorithm      5-2-4. gxvalid auto dialect detection algorithm
456        -----------------------------------------------      -----------------------------------------------
457        The  first  16bit  of kern table is enough to sense the version:  
458          - if first 16bit is 0x0000,        The first 16  bits of the `kern' table are  enough to identify the
459            kern table is in classic Apple-dialect        version:
460                          or classic Microsoft-dialect,  
461          - if first 16bit is 0x0001, and next 16bit is 0x0000,          - if  the first  16  bits are  0x0000,  the `kern'  table is  in
462            kern table is in new Apple-dialect.            classic Apple dialect or classic Microsoft dialect
463        If kern table is classic version, 16bit coverage is checked for in          - if the first 16 bits are  0x0001, and next 16 bits are 0x0000,
464        next. For first,  the coverage is decoded by classic Apple-dialect            the kern table is in new Apple dialect.
465        as following  (it is based on DumpKERN output):  
466          If the `kern'  table is a classic one,  the 16bit `coverage' field
467          is checked next.   Firstly, the coverage bits are  decoded for the
468          classic Apple dialect using the following bit masks (this is based
469          on DumpKERN output):
470    
471          0x8000: 1=horizontal, 0=vertical          0x8000: 1=horizontal, 0=vertical
472          0x4000: not used          0x4000: not used
473          0x2000: 1=cross-stream, 0=normal          0x2000: 1=cross-stream, 0=normal
474          0x1FF0: reserved          0x1FF0: reserved
475          0x000F: subtable format          0x000F: subtable format
476        If   any   of   reserved  bits  are  set  or  subtable  format  is  
477        interpreted  as 1 or 3,  we  take it  as  "impossible in classic        If  any  of  reserved  bits  are  set  or  the  subtable  bits  is
478        Apple-dialect", and retry by classic Microsoft-dialect.        interpreted as format 1 or 3, we take it as `impossible in classic
479          Apple dialect' and retry, using the classic Microsoft dialect.
480    
481          The most popular coverage in new Apple-dialect:         0x8000,          The most popular coverage in new Apple-dialect:         0x8000,
482          The most popular coverage in classic Apple-dialect:     0x0000,          The most popular coverage in classic Apple-dialect:     0x0000,
483          The most popular coverage in classic Microsoft dialect: 0x0001.          The most popular coverage in classic Microsoft dialect: 0x0001.
484    
485      5-3. Tested fonts    5-3. Tested fonts
486      -----------------    -----------------
487      We  checked  59  fonts  bundled  to  MacOS  which includes kern, and  
488      38 fonts bundled to Windows which includes kern.      We checked  59 fonts  bundled with MacOS  and 38 fonts  bundled with
489        - fonts bundled to MacOS      Windows, where all font include a `kern' table.
490          * new Apple-dialect      
491          - fonts bundled with MacOS
492            * new Apple dialect
493            format 0: 18            format 0: 18
494            format 2:  1            format 2:  1
495            format 3:  1            format 3:  1
496          * classic Apple-dialect          * classic Apple dialect
497            format 0: 14            format 0: 14
498          * classic Microsoft-dialect          * classic Microsoft dialect
499            format 0: 15            format 0: 15
500        - fonts bundled to Windows  
501          * classic Microsoft-dialect        - fonts bundled with Windows
502            * classic Microsoft dialect
503            format 0: 38            format 0: 38
504    
505      It looks strange that classic Microsoft-dialect fonts are bundled to      It looks strange that classic Microsoft-dialect fonts are bundled to
506      MacOS: they come from MSIE for MacOS, except of MarkerFelt.dfont.      MacOS: they come from MSIE for MacOS, except of MarkerFelt.dfont.
507    
508    
509    ACKNOWLEDGEMENT    ACKNOWLEDGEMENT
510    ---------------    ---------------
   Some part of gxvalid is derived from both gxlayout module and otvalid  
   module. Development of gxlayout was support of Information-technology  
   Promotion Agency(IPA), Japan.  
511    
512    The detailed analysis of undefined glyph ID utilization in mort, morx    Some parts of gxvalid are  derived from both the `gxlayout' module and
513    is provided by George Williams.    the `otvalid'  module.  Development of  gxlayout was supported  by the
514      Information-technology Promotion Agency(IPA), Japan.
515    
516      The detailed analysis of undefined  glyph ID utilization in `mort' and
517      `morx' tables is provided by George Williams.
518    
519  ------------------------------------------------------------------------  ------------------------------------------------------------------------
520    
# Line 437  Copyright 2004, 2005 by Line 522  Copyright 2004, 2005 by
522  suzuki toshiya, Masatake YAMATO, Red hat K.K.,  suzuki toshiya, Masatake YAMATO, Red hat K.K.,
523  David Turner, Robert Wilhelm, and Werner Lemberg.  David Turner, Robert Wilhelm, and Werner Lemberg.
524    
525  This  file  is  part  of the  FreeType  project, and may  only be  used,  This  file is  part  of the  FreeType  project, and  may  only be  used,
526  modified,  and  distributed  under  the  terms of  the FreeType  project  modified,  and  distributed under  the  terms  of  the FreeType  project
527  license, LICENSE.TXT.   By continuing to use, modify, or distribute this  license, LICENSE.TXT.  By continuing  to use, modify, or distribute this
528  file you  indicate that  you have  read the  license and understand  and  file  you indicate that  you have  read the  license and  understand and
529  accept it fully.  accept it fully.
530    
531    

Legend:
Removed from v.1.1  
changed lines
  Added in v.1.2

savannah-hackers-public@gnu.org
ViewVC Help
Powered by ViewVC 1.1.26