bugGNU roff - Bugs: bug #57506, Suspicious "slant"...

 
 

bug #57506: Suspicious "slant" values in devps/TI, devlbp/HI, devlbp/HBI

Submitter:  Dave <barx>
Submitted:  Mon 30 Dec 2019 04:23:19 PM UTC
   
 
Category:  Font devps Severity:  2 - Minor
Item Group:  Documentation Status:  Need Info
Privacy:  Public Assigned to:  gbranden
Open/Closed:  Open Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Sun 02 Jul 2023 03:13:33 PM UTC, comment #14: 

comment #13:

> TrueType fonts allow little scripts within the font to be
> called before a glyph is stroked, so I suspect this is what
> is happening to correct the glyph position.


Thanks for the detective work!  That explanation makes sense.

> Of course, when fontforge converts the font to type 1, any
> scriptlets in the font get dropped.


I suppose these scripts can be of arbitrary complexity, but I wonder if there's a way for fontforge to detect ones that merely alter a glyph's position, then account for that in the glyph definition it outputs.

> What a good reason for gropdf to support otf/ttf fonts natively!!


Surely just an afternoon's work... ;-}

> And now for something completely different...
>
> Assuming you have called your Libertine font family "Lib", try this:-


> echo "\fI\s'200'\v'200p'\N[2364]" | ./test-groff -Tpdf -fLib -ms > T.pdf


> Surprised he hasn't an italic lean!


I've learned that the numeric values that get placed in the font description file (the thing that \N[] is looking up) can vary--depending on, I'm not sure what, maybe different versions of fontforge or afmtodit, but I've run Peter's install-font.sh on the same .ttf files at different times and ended up with different numberings.  So I'm not sure what glyph \N[2364] maps to on your installation.  For me:

# fgrep 2364 /usr/share/groff/site-font/devps/LI
u1E9F   461,699,13,119,-19,75   2       2364    uni1E9F

U+1E9F is LATIN SMALL LETTER DELTA, and when I run your command, I indeed get an italic lowercase delta.  But I bet \N[2364] is an entirely different glyph in your Libertine Italic font.

Dave <barx>
Group Member
Sun 02 Jul 2023 12:51:35 PM UTC, comment #13: 

I wanted to find out why the position of the diaeresis was so different between Tinos and Libertine, given that the Z files were identical and the font entries are similar, both have a zero width. So, it must be something different in the type 1 fonts themselves, below are the disassembled glyph definitions from both fonts.

TinosI

u0308   0,632,0,136,136 2       1384    uni0308
/uni0308 {
        205 0 hsbw
        555 1231 rmoveto
        288 callsubr
        -356 hmoveto
        87 callsubr
        endchar
        } ND

LibertineI

u0308   0,620,0,83,271,75       2       2004    uni0308
/uni0308 {
        -451 0 hsbw
        1174 vmoveto
        136 callsubr
        328 hmoveto
        129 callsubr
        endchar
        } ND

Notice the left side bearing in the Libertine font is negative compared to the Tinos version, I think this is the reason Libertine gets it right. Interestingly, if I use the original Tinos ttf font in libreoffice, it gets the position of the diaeresis spot on. TrueType fonts allow little scripts within the font to be called before a glyph is stroked, so I suspect this is what is happening to correct the glyph position. Of course, when fontforge converts the font to type 1, any scriptlets in the font get dropped.

What a good reason for gropdf to support otf/ttf fonts natively!!

hsbw

sbx wx hsbw (13)
sets the left sidebearing point at (sbx, 0) and sets the character
width vector to (wx, 0) in character space. This command also
sets the current point to (sbx, 0), but does not place the point in
the character path. Use rmoveto for the first point in the path.
The name hsbw stands for horizontal sidebearing and width; hor-
izontal indicates that the y component of both the sidebearing
and width is 0. Either sbw or hsbw must be used once as the first
command in a character outline definition. It must be used only
once. In non-marking characters, such as the space character, the
left sidebearing point should be (0, 0).

And now for something completely different...

Assuming you have called your Libertine font family "Lib", try this:-

echo "\fI\s'200'\v'200p'\N[2364]" | ./test-groff -Tpdf -fLib -ms > T.pdf

Surprised he hasn't an italic lean!

Deri James <deri>
Group Member
Sat 01 Jul 2023 12:11:18 AM UTC, comment #12: 

comment #11:

> Your utf8 example has the diacritical mark above the n.


That's the expected result.  From http://en.wikipedia.org/wiki/Combining_character : "In Unicode, diacritics are always added after the main character."

> If I run it as a pdf using Tinos fonts the mark is above the a!


Stranger still, if I edit your PDF grout to change the "font" line to one I have on my system (Linux Libertine), then run the results through gropdf, I get a PDF with the mark above the n!  (See attached.)  So the fonts themselves seem to interpret the combining mark differently, since the grout is otherwise identical.

> The "slant" value in the TI font comes from the ItalicAngle
> parameter in the corresponding .afm file, it documents the
> angle of lean which was used when the font was designed,
> changing it will have no effect.


afmtodit has a command-line switch to specify a value for "slant" different from the .afm file's ItalicAngle.  The passage of afmtodit(1) quoted in comment #6 claims groff uses this value to adjust diacritic placement, but I've been unable to prove this.  (But the Libertine result shown in the attachment here sure could use that adjustment.)

(file #54904)

Dave <barx>
Group Member
Fri 30 Jun 2023 09:48:45 PM UTC, comment #11: 

Your utf8 example has the diacritical mark above the n. If I run it as a pdf using Tinos fonts the mark is above the a!

[derij@pip build (new-gropdf)]$ echo '\fIThis is Spin\[u0308]al Tap' | ./test-groff -Tpdf -fTINO -ms -P-e -Z
x T pdf
x res 72000 1 1
x init
p1
x font 41 TINOI
f41
s10000
V84000
H72000
md
DFd
tThis
wh2500
tis
wh2500
tSpin   <- current xpos after the n
Cu0308  <- special char - no move
tal     <- overprint a
wh2500
tT
H133570
tap
n12000 0
V768000
H540000
n12000 0
x trailer
V792000
x stop

Contrast with:-

[derij@pip build (new-gropdf)]$ echo '\fIThis is Spin\[u0308]al Tap' | ./test-groff -Tutf8 -fTINO -ms -P-e -Z
x T utf8
x res 240 24 40
x init
p1
x font 2 I
f2
s10
V280
H0
md
DFd
tThis
wh24
tis
wh24
tSpin
Cu0308
h24
tal
wh24
tTap
n40 0
V2560
H1560
n40 0
x trailer
V2640
x stop

I have never looked at grotty so I don't know what is going on.

The "slant" value in the TI font comes from the ItalicAngle parameter in the corresponding .afm file, it documents the angle of lean which was used when the font was designed, changing it will have no effect. You can adjust the italic adjustment which groff uses by fiddling with the numbers in the TI font (see groff_font(7) for the details).


(file #54903)

Deri James <deri>
Group Member
Fri 30 Jun 2023 08:00:22 PM UTC, comment #10: 

comment #6:

> This could be tested, but it's not obvious how: the "is used by
> groff" line above is awfully vague about what accent-placing
> mechanism uses the value.  The \o escape, for instance,


I tested this using a character that doesn't have a precomposed form in any font on my system, so groff can't fall back to that behind the scenes; it has to combine the letter and the diacritic.

As predicted, "slant"'s parameter has no effect on \o's glyph placement:

echo '\fIThis is Spi\o@n\[u00A8]@al Tap' | groff

This produces the same output no matter what "slant" value you put in font description file "TI".

The next thing I tried was a Unicode combining form of the diacritic glyph.  However, none of the typesetting fonts that ship with groff contain these characters.

$ echo '\fIThis is Spin\[u0308]al Tap' | groff > /dev/null
troff:<standard input>:1: warning: special character 'u0308' not defined

(This works for utf8 output, however.

$ echo '\fIThis is Spin\[u0308]al Tap' | groff -Tutf8 -P-i | cat -s
This is Spin̈al Tap

No telling how your browser will render that output line, though.)

I do have a locally installed italic font that contains U+0308, and using that makes the above command line work.  But as before, changing the "slant" parameter in the font description file has no effect on the placement of the ¨.

So I'm still in the dark about what mechanism afmtodit(1)'s "is used by groff" refers to.

Dave <barx>
Group Member
Sun 25 Jun 2023 12:46:01 AM UTC, comment #9: 

comment #7:

> I think it's safe to say any font description file that has
> more than one "slant" line is in error.


...unless it isn't.  Comment #1 of bug #64342 suggests a possible intent behind this.  But if that is the case, there should be accompanying comments explaining it.  (Or, better, the file should use one of the (IMHO) superior methods that comment puts forth to include both values.)

Dave <barx>
Group Member
Fri 23 Jun 2023 08:19:31 PM UTC, comment #8: 

Following up to myself:

comment #6:

> Huh, so RTFMing suggests this value might be "erroneous" on
> purpose.  afmtodit(1) says:


To be fair, afmtodit(1) is not the most obvious place to look for this nugget.  A user perusing the font description file and wanting to know what its contents mean will probably first turn to (currently) section 6.2.2, Font Description File Format, of the Texinfo manual.  Herein, the entirety of the "slant" description is: "The font's glyphs have a slant of N degrees; a positive N slants in the direction of text flow."  The fact that this value might not accurately reflect the glyphs' slant, due to needing to align accent marks, should be stated here.

(For context, I first stumbled upon this issue when needing to sic \S on a glyph that didn't have an italic version, and wanted to use the same slant as the italic font this glyph was mixing with.  Thus, knowing that this parameter in the font file might not accurately reflect the font's glyphs is pertinent to readers of the file, not just to those generating such files.)

I'd submit a patch to address the above Texinfo omission, but I think it's important to also mention what mechanism uses this value for this purpose, and this is still murky to me.

> the "is used by groff" line above is awfully vague about what
> accent-placing mechanism uses the value.


Upon further thought, it might not even be strictly accurate: the font description file is device-specific, so it seems under the purview of the postprocessor, not of groff, to utilize this information.  (This assumes font data is appropriately encapsulated and thus not visible to groff itself.  But in practice I think the barriers are more porous than that.)  Or the "groff" in this phrase might be merely a shorthand for the pipeline, since groff calls a postprocessor by default unless the user explicitly requests otherwise.

> The \o escape, for instance, claims to (horizontally) center
> all overstruck glyphs


I should point out that this claim also comes from the Texinfo manual, which we now know is not comprehensive when it comes to "slant"-related matters.  (I still have not done any testing.)

Dave <barx>
Group Member
Fri 23 Jun 2023 04:15:16 AM UTC, comment #7: 

While we're cataloguing suspicious slant values, I think it's safe to say any font description file that has more than one "slant" line is in error.

$ grep -c ^slant font/dev*/* | grep -v ':[01]$'
font/devlbp/HBI:2
font/devlbp/HI:2
$ grep ^slant font/devlbp/HI font/devlbp/HBI
font/devlbp/HI:slant 18.49
font/devlbp/HI:slant 0
font/devlbp/HBI:slant 18.49
font/devlbp/HBI:slant 0

For both files, both "slant" lines were added in the same commit (the one that originally added the files themselves):

$ git blame font/devlbp/HI | fgrep slant
7022ec8fcc (Werner LEMBERG 2000-02-27 01:41:16 +0000   1) slant 18.49
7022ec8fcc (Werner LEMBERG 2000-02-27 01:41:16 +0000   6) slant 0
$ git blame font/devlbp/HBI | fgrep slant
7022ec8fcc (Werner LEMBERG 2000-02-27 01:41:16 +0000   1) slant 18.49
7022ec8fcc (Werner LEMBERG 2000-02-27 01:41:16 +0000   6) slant 0

These are both italic fonts, so the "slant 0" line must be the bogus one.  (Unless 0 was the experimentally determined value that makes accents align correctly...)

Dave <barx>
Group Member
Fri 23 Jun 2023 12:14:30 AM UTC, comment #6: 

Huh, so RTFMing suggests this value might be "erroneous" on purpose.  afmtodit(1) says:

"the slant ("angle") parameter in the font description file...is used by groff in the positioning of accents.  By default afmtodit uses the negative of the ItalicAngle specified in the AFM file; with true italic fonts it is sometimes desirable to use a slant that is less than this.  If you find that an italic font places accents over base glyphs too far to the right, use -a to give it a smaller slant."

I'm not completely convinced this is the reason for the slant discrepancy: The slant of the TI and TBI fonts look basically identical, suggesting that if one value required fudging, presumably both would.

This could be tested, but it's not obvious how: the "is used by groff" line above is awfully vague about what accent-placing mechanism uses the value.  The \o escape, for instance, claims to (horizontally) center all overstruck glyphs, which would disallow slant-based adjustments.  Further research is required.

Dave <barx>
Group Member
Sun 23 May 2021 07:52:08 AM UTC, comment #5: 

comment #4:

> I wondered there (and haven't investigated any further to try to answer
> myself) whether there's a checklist of updates to run for each release.


No, as of when I wrote that.

Yes, as of commit 0907598d.

Dave <barx>
Group Member
Fri 14 Aug 2020 11:47:23 PM UTC, comment #4: 

You raise important questions, to which I don't know any answers.  You demonstrate more font knowledge in comment #2 than I have in my entire brain.

comment #2:

> (1) It's important to determine whether these files are purely
> generated by afmtodit or whether they undergo hand-editing afterwards.


A question also relevant to bug #58894 and bug #58897.

Werner probably knows the answer.

http://lists.gnu.org/archive/html/groff/2020-08/msg00024.html suggests that John Gardner knows a thing or two about fonts as well.

> Either way, this procedure should become part of the  build
> or release-management process (depending on the tedium level).


Bug #57594 also concerns an update that should be done every release cycle.  I wondered there (and haven't investigated any further to try to answer myself) whether there's a checklist of updates to run for each release.

> extract metrics from fonts, then rewrite the metrics in ditroff,
> possibly with fixups like the one requested in this ticket.


One would ideally hope the original font files contain correct data, or that if they don't, bugs can be reported somewhere to get them corrected, so that we don't have to patch font metrics for every release in perpetuity.

Dave <barx>
Group Member
Fri 14 Aug 2020 12:25:28 PM UTC, comment #3: 


> (4) Do we need to do something similar for the basic 14 PDF fonts?


As part of install these are copied from the font/devps directory.


Deri James <deri>
Group Member
Fri 14 Aug 2020 09:17:23 AM UTC, comment #2: 

It seems to me that these files should be regenerated, if not on every build, then at least for every release.

However, given that the original font files whose metrics are taken are proprietary Adobe fonts (correct me if I'm wrong), that creates kind of a sticky wicket from a Debian software-freedom perspective.  It's the sort of thing that knocks a package from "main" into "contrib" (because at build or runtime it depends on non-free stuff).

The good news is while 20 years ago, popular opinion was that freely-licensed fonts were an unrealistic concept because font design was too specialized, too unrewarding, or too hard for anyone to do it under a FLOSS license, that wisdom has been exposed as the fallacy it always was.

I have a vague understanding that there exist "metrically-equivalent" fonts for all sorts of popular proprietary fonts.  If there are ones for the devps fonts, then by definition it should be impossible to tell from which fonts the metrics were extracted.

This also raises the possibility of simply shipping the free fonts with groff and installing them, though this might be superfluous as they're surely packaged and managed elsewhere.

I feel like I know a little but not quite enough to move this bug forward.

(1) It's important to determine whether these files are purely generated by afmtodit or whether they undergo hand-editing afterwards.  Either way, this procedure should become part of the  build or release-management process (depending on the tedium level).

(2) Do free replacements all of the devps fonts exist?  Even if not, one can argue against the "contribification" of groff within Debian by observing that the metrics extracted from such fonts are not encumbered by copyright; if they were, the creation of unlicensed replacements as Free Software would not have been possible in the first place.  It's furthermore my understanding, that, at least in the United States, copyright resides in digital fonts only insofar as they effectively contain programs to perform hinting.  So bitmapped (raster) and outline fonts are not copyrightable.

(3) If the answer to (2) is no, we should be able to just extract the metrics from the Adobe fonts in some convenient format, keep that in the source tree, and then perform transforms on that.  Right?  It seems like this would break the function of afmtodit in half--extract metrics from fonts, then rewrite the metrics in ditroff, possibly with fixups like the one requested in this ticket.  Except maybe this is already done--AFM is after all the format that afmtodit reads, and in the enscript package on my Debian system I see .afm files that seem to match most, albeit not all, of the fonts we have in devps.

(4) Do we need to do something similar for the basic 14 PDF fonts?

(5) The end result I envision, if several of my assumptions above are correct, would look like this:

A. Keep Free, Adobe-compatible PFA fonts, or metric data extracted from non-Free Adobe PFA fonts in the source tree, in the form of .afm files.
B. Run afmtodit on these.
C. Run ed or sed scripts to apply corrections or improvements.
D. Ship the result as we do today.

In conclusion, the the fixup envisioned here, as well as any others, like adding space kerning information where the fonts don't provide it themselves, would become part of step C above.

This font stuff gets me pretty far out of my comfort zone so please correct all misconceptions I have above.

G. Branden Robinson <gbranden>
Group administrator
Tue 17 Mar 2020 08:39:21 AM UTC, comment #1: 

This bug dates to at least groff 1.02: commit 351da0dc, the oldest available in git, shows that both of these files (then in directory ps/devps) had the same slant values then that they have today.


$ git show 351da0dc:ps/devps/TI | grep ^slant
slant 7
$ git show 351da0dc:ps/devps/TBI | grep ^slant
slant 15


But these files, while they are groff source files (in that they live in the groff source-code repository), do not appear to be actual source files, in that they are built by font/devps/generate/Makefile (today, or ps/devps/FontMakefile back in commit 351da0dc).  Thus this bug may be in the source files that TI and TBI are built from, or in the tools used to build them.

The primary tool is the afmtodit Perl script, which is part of groff (src/utils/afmtodit/afmtodit.pl, or ps/devps/afmtodit back in 351da0dc).

I am not sure of the origin of the primary source files, Times-Italic.afm and Times-BoldItalic.afm.  They don't live in the groff source tree (if they did, presumably TI and TBI would not, instead being built by the groff build process) or anywhere on my system.  The Makefile expects them to be in /usr/local/lib/afm.

Dave <barx>
Group Member
Mon 30 Dec 2019 04:23:19 PM UTC, original submission:  

The output of this command, run from the top of the groff source tree (or from /usr/share/groff/current on a system with groff installed) is highly questionable:

$ grep ^slant font/devps/T*
font/devps/TBI:slant 15
font/devps/TI:slant 7

It surely isn't correct that Bold Italic has twice the slant of Italic for the ps device.  The slants appear visually identical (as they should), and the slant values are identical in every other pair of Italic / Bold Italic fonts in font/devps (grep ^slant font/devps/*), and nearly identical for Times Italic and Times Bold Italic in every other font/dev* directory (grep ^slant font/dev*/T*).

Dave <barx>
Group Member

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #54904:  SpinalTapLibertine.pdf added by barx (507KiB - application/pdf)
file #54903:  SpinalTap.pdf added by deri (10KiB - application/pdf)

 

Depends on the following items: None found

Digest:
   bug dependencies.

 

Carbon-Copy List
  • -email is unavailable- added by deri (Posted a comment)
  • -email is unavailable- added by gbranden (Posted a comment)
  • -email is unavailable- added by barx (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 7 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2024-01-02 gbranden Dependencies- bugs #58894 is dependent
    2023-07-01 barx Attached File- Added SpinalTapLibertine.pdf, #54904
    2023-06-30 deri Attached File- Added SpinalTap.pdf, #54903
    2023-06-23 barx SummaryErroneous &quot;slant&quot; value in Times italic font(s) for ps device Suspicious "slant" values in devps/TI, devlbp/HI, devlbp/HBI
    2021-05-23 barx Severity3 - Normal 2 - Minor
    2020-08-14 gbranden StatusNone Need Info
        Assigned toNone gbranden

    Back to the top

    Powered by Savane 3.12.
    Corresponding source code