bugGNU troff - Bugs: bug #63354, Refine fallbacks.tmac

 
 

bug #63354: Refine fallbacks.tmac

Submitter:  Dave <barx>
Submitted:  Sat 12 Nov 2022 11:06:51 PM UTC
 
Category:  Macro - others/general Severity:  2 - Minor
Item Group:  Rendering/Cosmetics Status:  Need Info
Privacy:  Public Assigned to:  None
Open/Closed:  Open Planned Release:  None
* Mandatory Fields

Add a New Comment (Rich Markup)
   

Comment Type & Canned Response:
       No canned response available

 

( Jump to the original submission )

Thu 29 Dec 2022 02:16:48 AM UTC, comment #27: 

Hi Dave,

comment #24:

> > And if you're right for troff devices that Symbol is always there
> > and it always defines this glyph, it can't degrade typography.
>
> This forgets the lesson I had to learn the hard way (see comment #3) that .fchar definitions take precedence over special fonts.


Ah, yes.  I seem to keep forgetting that, too.  I have visions of a `pchar` request that takes a character as an argument and makes the formatter report how it resolves the damn thing.  Making people read through the eight-step procedure documented in section 5.19.4 of our Texinfo manual--which I haven't reviewed for correctness, incidentally--is simply sadistic.

Nevertheless `fchar` seems to be the right request, rather than `schar`, because a hyphen will nearly(?) always be something that is rendered adjacent to glyphs from a text (potentially styled) font.

>  So as things stand, the definition in fallbacks.tmac will always hide the Symbol font's u2026, degrading typography when using fonts that lack their own u2026.


(U+2026 is the ellipsis.)

But does that happen?  Do people ship text fonts that lack ellipses?

The PostScript Language Reference manual, 3e, suggests that text fonts are reliable in providing ellipses.  The Symbol font seems to have one too (pp. 775-778).

>  And this hiding seems to happen even with the lowest-precedence member of the .*char family, .schar: given the following input, groff uses the "schar" definition rather than falling back to the u2026 in Symbol:
>
> .schar \[u2026] FOO
> .ft ZD
> \[u2026]


Agreed.  That much at least is consistent with §5.19.4.  If I add a `special` request, troff locates "S"'s ellipsis again.

$ cat EXPERIMENTS/zapf-hyphen.groff
.\" troff -R
.special S SS ZD
.schar \[u2026] FOO
A long time ago in a galaxy far, far away\[u2026]
$ troff -R EXPERIMENTS/zapf-hyphen.groff | grep -C3 u2026
tw
H226060
tay
Cu2026
h10000
n12000 0
x trailer

 

> > You might however be forgetting grodvi and grolbp, which don't
> > offer the PS Symbol font.
>
> Indeed I was.  And offhand I'm not sure how to accommodate those without also degrading PostScript.


Possibly the device tmac files should be making `special` requests, though I'm a little leery of this since it may be contrary to the request's initial purpose.  There is already a CSTR #54 procedure for searching special fonts.

This is not a question I feel can be resolved for groff 1.23.0.

> > On nroff devices, users of "ascii" and "latin1" devices
> > actually come out ahead of "utf8" users.
>
> Luckily, we can easily test for terminal devices and give them different fallbacks.


I'm not inclined to change things further in this respect at present; I may think a half-width character cell ellipsis is ugly as hell, but I'm uneasy with going out of my way to "protect" the user from it.

> For grodvi and grolbp, all I can do is observe that (a) terminal, PostScript and PDF are likely more important output formats for a vast majority of users, and (b) grodvi and grolbp had no u2026 fallback in prior releases, so lacking one in 1.23 isn't a regression.


I have no real argument with this, I am simply trying to think in  maintainerly, comprehensive way to serve all of groff's users, even those that use exotic output devices.

> Thus I'm starting to think the best solution is to make the u2026 fallback terminal-only for 1.23.  But if there's a way to keep the PostScript/PDF behavior as it was in 1.22.4 but also improve other non-terminal output, I'm open to that.


comment #25:

> If fallbacks.tmac gets tweaked before 1.23 (or perhaps even before rc2), one more trivial change I'd recommend.  Two lines in the file currently read:

> .\" \[u2010] is always defined thanks to uniglyph.cpp.
> .\"fchar \[u2010] -\:\" hyphen

> The first of these lines is worthwhile, in case someone comes along later and is tempted to add a \[u2010] definition.  But the second seems misleading: point (b) of comment #8 illustrates that even were the uniglyph.cpp definition removed or circumvented, the fchar here wouldn't do what was intended.  So it's a little dubious to put it in a shipped .tmac file, even in commented form, as it implies to anyone reading the code that the construction could work.
>
> So my preference would be to see the second line removed entirely.  But this is certainly not a blocker, since it's inactive code.


There are so many loose threads tugged on by this baby step toward Unicode support, as if a toddler's leg is wrapped with several strands of dental floss, each connected distally to a different precariously balanced piece of fine china that I feel I must simply solicit a patch from you at this point.

I haven't lost my spirit to fight with these issues in general, but I feel extremely constrained from whacking at them within the parameters I've imposed on myself for getting to a 1.23.0 release.

G. Branden Robinson <gbranden>
Project Administrator
Thu 29 Dec 2022 02:14:09 AM UTC, comment #26: 

  About the character "figure dash".

  The total width of a figure dash must be the same as that of a digit,
so (including "air")

.if !c \[u2012] \{\
.  nr lfiguredash \w'0'u-(2*\w'\^'u)
.  char \[u2012] \^\v'-0.3m'\D'l \n[lfiguredash]u 0m'\v'0.3m\^'
.  rm lfiguredash
.\}

  And with a synonym (PostScript name)

.tr \[figuredash]\[u2012]

Bjarni Ingi Gislason <bjarniig>
Thu 29 Dec 2022 12:53:36 AM UTC, comment #25: 

If fallbacks.tmac gets tweaked before 1.23 (or perhaps even before rc2), one more trivial change I'd recommend.  Two lines in the file currently read:

.\" \[u2010] is always defined thanks to uniglyph.cpp.
.\"fchar \[u2010] -\:\" hyphen

The first of these lines is worthwhile, in case someone comes along later and is tempted to add a \[u2010] definition.  But the second seems misleading: point (b) of comment #8 illustrates that even were the uniglyph.cpp definition removed or circumvented, the fchar here wouldn't do what was intended.  So it's a little dubious to put it in a shipped .tmac file, even in commented form, as it implies to anyone reading the code that the construction could work.

So my preference would be to see the second line removed entirely.  But this is certainly not a blocker, since it's inactive code.

Dave <barx>
Project Member
Wed 28 Dec 2022 11:39:54 PM UTC, comment #24: 

Concerning U+2026:

comment #19:

> The (PostScript) Symbol font won't be there on nroff devices.


A valid point that I hadn't considered.

> And if you're right for troff devices that Symbol is always there
> and it always defines this glyph, it can't degrade typography.


This forgets the lesson I had to learn the hard way (see comment #3) that .fchar definitions take precedence over special fonts.  So as things stand, the definition in fallbacks.tmac will always hide the Symbol font's u2026, degrading typography when using fonts that lack their own u2026.  And this hiding seems to happen even with the lowest-precedence member of the .*char family, .schar: given the following input, groff uses the "schar" definition rather than falling back to the u2026 in Symbol:

.schar \[u2026] FOO
.ft ZD
\[u2026]

> You might however be forgetting grodvi and grolbp, which don't
> offer the PS Symbol font.


Indeed I was.  And offhand I'm not sure how to accommodate those without also degrading PostScript.

> On nroff devices, users of "ascii" and "latin1" devices
> actually come out ahead of "utf8" users.


Luckily, we can easily test for terminal devices and give them different fallbacks.

For grodvi and grolbp, all I can do is observe that (a) terminal, PostScript and PDF are likely more important output formats for a vast majority of users, and (b) grodvi and grolbp had no u2026 fallback in prior releases, so lacking one in 1.23 isn't a regression.

Thus I'm starting to think the best solution is to make the u2026 fallback terminal-only for 1.23.  But if there's a way to keep the PostScript/PDF behavior as it was in 1.22.4 but also improve other non-terminal output, I'm open to that.

Dave <barx>
Project Member
Wed 28 Dec 2022 10:18:47 PM UTC, comment #23: 

[U+2012 thoughts continued]

http://en.wikipedia.org/wiki/Dash#Figure_dash suggests an en dash as a figure dash replacement.  To be conservative (in case the user's font has an en dash wider than its figures), we might also define a glyph that is a hyphen centered within a figure space.  This produces decent results with the sample snippet from comment #22:

.fchar \[u2012] \o'\0-'

In Times, even using \[en] in place of the - here feels too crowded, though less so than the raised \[ru].  The hyphen falls more on the too-short side, but I contend this is the better side to fall on; this is, after all, only a fallback glyph, for fonts lacking a U+2012 designed to interact pleasingly with its figures.

(The order of the glyphs in the \o escape shouldn't matter, since this definition isn't used for terminal devices, but it is crafted to degrade well in non-overstriking situations.)

Dave <barx>
Project Member
Wed 28 Dec 2022 08:04:36 PM UTC, comment #22: 

comment #18:

> Re: #6, U+2012, I think the glyph just needs a little elbow room.
>
> Try this.


That improves the aesthetics--but also torpedoes an essential property of this glyph, that its width be the same as that of a digit.  Observe how these digits no longer align vertically:

printf '7\[u2012]5 6\[u2012]0\n.br\n735 690\n' | groff

Dave <barx>
Project Member
Wed 28 Dec 2022 09:31:59 AM UTC, comment #21: 

Step back down to minor severity (not normal) after its temporary promotion for release-critical items.

G. Branden Robinson <gbranden>
Project Administrator
Tue 27 Dec 2022 09:25:36 AM UTC, comment #20: 

The 1.23.0-critical(?) stuff is pushed.

Dropping to normal severity.

commit fa1553c33ecfe925cc0c0eac41a30903ae040987
Author: G. Branden Robinson <g.branden.robinson@gmail.com>
Date:   Mon Dec 26 12:44:32 2022 -0600

    [tmac]: Revise "fallback" character definitions.

    ...in several respects for Unicode characters from General Punctuation
    block.

    * tmac/fallbacks.tmac: Comment out fallbacks for U+200B, U+2010, U+2011,
      and U+201[89CD], due to ineffectuality or apparent groff bugs.
      Redefine U+2012 fallback to include hair space (\^) around synthetic
      figure dash.  Redefine U+2016 to use `\[ba]` special characters
      instead of ordinary `|` characters to dodge possible character
      translations by user.

    Fixes part of <https://savannah.gnu.org/bugs/?63354>.  Thanks to Dave
    Kemper for the (multifarious) discussion.

G. Branden Robinson <gbranden>
Project Administrator
Mon 26 Dec 2022 06:45:12 AM UTC, comment #19: 

comment #15:

> 0 permanently remove \[u2026] (comment #6)


> 0 Should be done before 1.23: only degrades typography without any upside in real-world usage, since Symbol will always be there.


The (PostScript) Symbol font won't be there on nroff devices.  And if you're right for troff devices that Symbol is always there and it always defines this glyph, it can't degrade typography.

You might however be forgetting grodvi and grolbp, which don't offer the PS Symbol font.  grolbp in fact doesn't offer any special fonts at all.  (Technically, they're all special.  ¯\_(ツ)_/¯ )

On nroff devices, users of "ascii" and "latin1" devices actually come out ahead of "utf8" users.

$ printf 'foo\\[u2026]bar\n' | tgu | cat -s
foo…bar
$ printf 'foo\\[u2026]bar\n' | tg -Tascii | cat -s
foo...bar
$ printf 'foo\\[u2026]bar\n' | tg -Tlatin1 | cat -s
foo...bar

So I'm inclined to keep this one.

G. Branden Robinson <gbranden>
Project Administrator
Mon 26 Dec 2022 06:32:43 AM UTC, comment #18: 

Re: #6, U+2012, I think the glyph just needs a little elbow room.

Try this.

-.  fchar \[u2012] \v'-.3m'\l'\w"\0"u\[ru]'\v'+.3m'\" figure dash
+.  fchar \[u2012] \^\v'-.3m'\l'\w"\0"u'\v'+.3m'\^\" figure dash

G. Branden Robinson <gbranden>
Project Administrator
Mon 26 Dec 2022 06:19:59 AM UTC, comment #17: 

Dave says there are 1.23.0-critical issues here, so kicking this up to Important until they are taken care of.

G. Branden Robinson <gbranden>
Project Administrator
Mon 26 Dec 2022 06:19:30 AM UTC, comment #16: 

The explanation for the first half of comment #8 appears to be this.

https://git.savannah.gnu.org/cgit/groff.git/tree/src/libs/libgroff/uniglyph.cpp?id=59efbfc364daee11af9207a3fc79675b6f5bfc25#n334

Normally, I get this:

$ printf '.fchar \\[u2010] BOGOSITY\nfoo\\[u2010]bar\n.pl \\n[nl]u\n' | ./build/test-groff -Tps -a
<beginning of page>
foo<hy>bar

But if I change it to something obviously wrong, I get this:

$ printf '.fchar \\[u2010] BOGOSITY\nfoo\\[u2010]bar\n.pl \\n[nl]u\n' | ./build/test-groff -Tps -a
<beginning of page>
foo<sl>bar

And if I comment it out, I get this:

$ printf '.fchar \\[u2010] BOGOSITY\nfoo\\[u2010]bar\n.pl \\n[nl]u\n' | ./build/test-groff -Tps -a
<beginning of page>
foo<u2010>bar

N.B., groff -a output does not "chase" fallback character definitions of the sort we use in "fallback.tmac".

G. Branden Robinson <gbranden>
Project Administrator
Sat 24 Dec 2022 06:05:07 AM UTC, comment #15: 

Yeah, this ticket is chockablock with to-do items, but they're sprinkled across several comments and interspersed with a couple of blind alleys.  So here's a checklist of tasks.

  1. reorder file (original submission)
  2. consistently comment file (original submission)
  3. comment out \[u200B] until bug #58958 resolved (comment #5)
  4. permanently remove \[u2010] (comment #8)
  5. potentially defer \[u2011] until bug #63360 resolved (comment #10)
  6. fix \[u2012] (original submission; correction in comment #13)
  7. refine \[u2016] definition for robustness (original submission)
  8. comment out \[u2018], \[u2019], \[u201C], and \[u201D] until bug #59932 resolved (comment #11)
  9. permanently remove \[u2026] (comment #6)
  10. add a \[u202F] fallback (comment #4)
  11. give \[u203D] an nroff fallback (comment #9, as amended by comment #12)
  12. refine \[u2052] definition for robustness (original submission)

Every one of these involves an alteration to commit 132182bd, which was added since 1.22.4, so for 1.23 purposes, the pertinent question is: which changes make things better as-is, and which make things worse, since the last release?

Obviously, "better" and "worse" are subjective.  But I'm gonna go ahead and pretend they aren't, and further pretend that I'm the final arbiter, and pass judgment on every item above.

  1. Refactor only.  Irrelevant to 1.23.
  2. Refactor only.  Irrelevant to 1.23.
  3. Should be done before 1.23, since the the current definition is broken, and groff has a long-established escape for doing this.
  4. Not important to 1.23 (though OTOH removing the line is trivial so why not).
  5. Should be done before 1.23: while it's a shame to not be able to support U+2011, the subpar rendering is worth addressing before it's unleashed onto the world.
  6. The fix to this is (probably) not trivial, but the current rendering is ugly, so this should be commented out before 1.23.
  7. Should be done before 1.23: fix is simple and avoids plausible real-world problems.
  8. Should be done before 1.23: these four have long-established groff aliases that do the right thing, while these fallbacks would degrade typography.
  9. Should be done before 1.23: only degrades typography without any upside in real-world usage, since Symbol will always be there.
  10. Not important to 1.23.
  11. Not important to 1.23.
  12. Probably not important to 1.23, mostly because there's no easy fix.  The potential pitfall to the user is a seemingly inexplicable character substitution, but U+2052 is obscure enough that I don't see much practical risk.

These opinions are correct and incontrovertible.  I have spoken.

Dave <barx>
Project Member
Tue 20 Dec 2022 03:47:09 AM UTC, comment #14: 

Hi Dave,

Is any part of this important to get into 1.23.0?  I fixed the only Savannah ticket that bore that as the "Planned Release", but there are many details in this one.

G. Branden Robinson <gbranden>
Project Administrator
Mon 19 Dec 2022 08:42:16 PM UTC, comment #13: 

original submission:

> The figure dash (\[u2016]) as currently designed works well with
> some figures but not others, in at least the TR font.  Consider
> the PostScript output of:
>
> echo '7\[u2012]5 6\[u2012]0' | groff


As the command implies, the figure dash is \[u2012], not \[u2016] as stated in the text above.

Dave <barx>
Project Member
Sun 11 Dec 2022 10:28:39 PM UTC, comment #12: 

Commit 230ad718 has mooted everything after the "HOWEVER" in comment #9.

Dave <barx>
Project Member
Wed 07 Dec 2022 09:48:44 PM UTC, comment #11: 

The point comment #10 made about \[u2011] equally applies to \[u2018] (left single quotation mark), \[u2019] (right single quotation mark), \[u201C] (left double quotation mark), and \[u201D] (right double quotation mark), all common characters that are frequently kerned next to certain uppercase letters.

I hate to take all these Unicode representations out of circulation until the thornier underlying problems can be solved.  However, they were in no previously released groff anyway, and at least with these four (unlike \[u2011]), groff has decades-established local aliases (\[oq], \[cq], \[lq], and \[rq]) that work correctly in all respects, so retaining the recently added, incorrectly kerned variants is of dubious value.

Dave <barx>
Project Member
Sun 04 Dec 2022 02:50:56 PM UTC, comment #10: 

And now... we should talk about U+2011, the character that has spilled the most ink over in bug #58930.

In a nutshell, the problem is that U+2011 is, by definition, visually identical to an ordinary hyphen (U+2010), a character which is frequently kerned next to several uppercase letters.  But as currently implemented, the U+2011 will never be kerned, because groff does not kern defined characters (bug #59932).  Bug #63360 documents the U+2011 problem, but points out that solving it is likely not simple, and thus can't be expected soon.

So the question is how to handle this in the meantime.  Much of the discussion in 58930 resolved to "punt on this issue and see if the users complain," which I agreed was reasonable in several edge cases.

I'm less comfortable with it in the U+2011 case, since the miskerning there (a) is much commoner in ordinary text, and (b) may not be something a user will complain about, per se, but that will subtly degrade the type quality--something the groff user might not even notice but a reader of that user's output might (from which we're unlikely to ever get a report back).

If the groff team decides this problem is worth addressing, the only reasonable short-term solution, as far as I can see, is to follow bug #63332's lead and comment out the \[u2011] line, marking it as requiring more R&D than can be presently dedicated to it.  (The corresponding comment should point to bug #63360, which is dedicated to this issue.)

Dave <barx>
Project Member
Thu 01 Dec 2022 09:43:43 AM UTC, comment #9: 

A possible refinement to U+203D:

The current definition is

.if t .fchar \[u203D] \o'?!'\" interrobang

This (theoretically) leaves the character undefined on the terminal--which it has been until recently, so not a huge deal--but as long as we're defining fallbacks anyway, the string "!?" is closest in spirit achievable without overstriking.  Thus the line could be:

.ie t .fchar \[u203D] \o'?!'\" interrobang
.el .fchar \[u203D] !?

HOWEVER.  Because of the ordering issue discussed in bug #63332, the ".if t" lines are not working correctly in nroff mode anyway, meaning that the character as currently defined is resulting in "!" being output on terminals.

I feel that, if we can only use one character to represent the interrobang, the "?" is a better choice than the "!" (as the "?" can change the meaning of a phrase, while the "!" typically only changes its level of emphasis).  So as long as fallbacks.tmac can't test what mode it's in, I'd recommend changing the order of the current definition, so that it reads:

.fchar \[u203D] \o'!?'\" interrobang

This won't change the typesetting at all, but will (IMHO) improve the terminal output a bit.

Dave <barx>
Project Member
Mon 21 Nov 2022 04:17:45 PM UTC, comment #8: 

The plot keeps thickening.  The \[u2010] definition should also be removed because (a) it doesn't seem to be invoked, and (b) it wouldn't work if it were.

(a): None of groff's fonts define a u2010 character.  But apparently earlier in the process, groff intercepts the u2010 and turns it into a hyphen, so the .fchar definition is never seen.  This can be demonstrated by changing the line in fallbacks.tmac to read, for example:

.fchar \[u2010] GARBAGE

Even with this change, groff outputs a hyphen for \[u2010]:

$ echo 'Boo\[u2010]yah.' | groff -Tascii | cat -s
Boo-yah.

The same results are in PostScript output, even though font TR does not contain a u2010 character.  So nothing is ever using this .fchar definition.

(b): Now let's avoid the u2010 name altogether and define a new character.

$ cat booyah
.fchar \[myhyphen] -\:
.ll 1p
Boo-yah.
Boo\[myhyphen]yah.
$ groff -Tascii booyah | cat -s
troff:booyah:3: warning [p 1, 0.0i]: cannot break line
troff:booyah:3: warning [p 1, 0.2i]: cannot break line
troff:booyah:4: warning [p 1, 0.3i]: cannot break line
Boo-
yah.
Boo-yah.

This looks like our old friend bug #58958 coming out to play.  So even if the u2010 definition in fallbacks.tmac were getting called, it wouldn't do what you wanted anyway.

Upshot: this line should be removed.  Because of point (a) above, it's not actually doing any harm, but ideally the shipped tmac files should be free of dead code.

Dave <barx>
Project Member
Mon 21 Nov 2022 04:18:13 AM UTC, comment #7: 

And in case anyone was concerned, U+2026 is the only \[u....] glyph in Symbol that fallbacks.tmac obscures:

$ egrep '^u[0-9A-F]{2}' font/devps/S | cut -f1,5
u2026   ellipsis
u2320   integraltp
u23AE   integralex
u2321   integralbt

Dave <barx>
Project Member
Mon 21 Nov 2022 12:29:57 AM UTC, comment #6: 

original submission:

> Every general-purpose font groff ships includes this character.
> Further, the Symbol font (which also includes it) being a default
> special font means that even users using a locally installed font that
> lacks a U+2026 will get it for free without doing any extra work.
>
> All this might argue for removing this definition from fallbacks.tmac
> entirely.


My realization in comment #3 that an .fchar definition supersedes a special-font glyph only reinforces the above: if the user's current font lacks a \[u2026], it is better to fall back to the well-formed one in Symbol than to the more problematic one defined in fallbacks.tmac.  Further, the fact that it's nigh impossible to remove Symbol as a special font means that a user would have to resort to an "ugly hack" of the type Branden describes in bug #63366 in order to ever be lacking a \[u2026]--and once someone is in this power-user territory, I think it's fine if they're working without a net.

Dave <barx>
Project Member
Sun 20 Nov 2022 08:04:54 PM UTC, comment #5: 

Another addendum:

Unicode's U+200B ZERO WIDTH SPACE has the semantics of groff's \: escape.  However, the current definition for \[u200B] in fallbacks.tmac (\h'0') does not have U+200B's essential breaking property:

$ cat u200B-test
.ll 1p
ab\:cd
ab\h'0'cd
$ nroff -Ww u200B-test | cat -s
ab
cd
abcd

And sadly, while bug #58958 is unresolved, we can't do the obvious ".fchar \[u200B] \:".  (.tr likewise doesn't work, though one wouldn't expect it to, since its definitions are supposed to be glyphs, which \: isn't.)  I have no other suggestions for how to tie this functionality to \[u200B].

So perhaps the best thing here is to comment out the \[u200B] definition, with an additional comment saying it's awaiting a fix to #58958.  In the meantime, \[u200B] will be unrecognized, but that's already the status quo (in 1.22.4, and in git until less than a month ago), and is less misleading than implementing it in a noncompliant way.  (I'm not sure how, or whether, \h'0' effectively differs from \&.)

Dave <barx>
Project Member
Wed 16 Nov 2022 11:40:23 AM UTC, comment #4: 

As an addendum to the list of fallbacks.tmac changes set out in the original submission, one more fallback character could be added to the file, one mentioned all the way back in the original submission of the beleaguered bug #58930: \[u202F] (NARROW NO-BREAK SPACE).

Groff has two nonbreaking thin spaces, \| and \^.  It is perhaps unclear which of these \[u202F] should map to, but either one would be an improvement over its current mapping to the warning "can't find special character `u202F'".  All the narrow spaces in groff are already nonbreaking, so nothing extra need be done for that.

Dave <barx>
Project Member
Tue 15 Nov 2022 07:25:04 PM UTC, comment #3: 

Um, you know that last comment where I said I was wrong?  Well, turns out I was wrong, and I was actually right.  (So, also wrong.)

What I missed is that whatever special fonts are in play have no bearing on the situation, because the clearly documented order of glyph-searching places .fchar definitions before special fonts.  So as long as the \[u2026] .fchar in fallbacks.tmac is in force, whatever is happening in special-font-land is irrelevant.

This is verifiable in practice.  Using the same input file from comment #2, this command run using the latest groff code produces this output:

$ groff -Z ellipsis_demo | sed -n '/font 11 S/,/x trailer/p'
x font 11 S
f11
V24000
H72000
t.
h1666
t.
h1666
t..
h1666
t.
h1666
t..
h1666
t.
h1666
t.
n12000 0
x trailer

Then I commented out the \[u2026] definition in fallbacks.tmac and ran it again.

$ groff -Z ellipsis_demo | sed -n '/font 11 S/,/x trailer/p'
x font 11 S
f11
V24000
H72000
Cu2026
h10000
Cu2026
h10000
Cu2026
h10000
n12000 0
x trailer

Here it's "easy" (if you're adept at reading intermediate output) to see that in the former case, groff is constructing the ellipsis from spaced periods, while in the latter, it's calling upon a character named u2026, which it's pulling from the current font, which is S.

This also explains why I was led astray in comment #2: I took the mere loading of font S as evidence that groff must be using that font's u2026--when in fact it was using that font's period, as a special-font fallback because the last requested font (ZD) also lacks a period glyph.

So please disregard comment #2.  The final example in the original submission is fine as-is, though the ".special" line is irrelevant (which is fine, since it doesn't do anything anyway: bug #63366 remains a valid point about special-font handling) and can be removed.

Dave <barx>
Project Member
Mon 14 Nov 2022 05:34:13 PM UTC, comment #2: 

One of my examples in the original submission doesn't do what I thought it did.

$ cat ellipsis_demo
.nf
Switch to a font without u2026, and remove the Symbol fallback.
.ft ZD
.special
\[u2026]\[u2026]\[u2026]
$ groff -Z ellipsis_demo | fgrep font
x font 5 TR
tfont
x font 11 S

I had intended for the ".special" line to zero out the list of special fonts, based on this sentence in the "Special Fonts" section of the manual: "Previous calls to 'special' or 'fspecial' are overwritten; without arguments, the particular list of special fonts is set to empty."

But the grout output shows that the Symbol font is still being invoked.  (And, notably, ZD is not, since after the request to switch to ZD, the input file asks only for one glyph, which ZD doesn't have.)

I opened bug #63366 to report the documentation issue.  As that bug report points out, there doesn't seem to be a way for a document to remove Symbol as a special font.  So to get this example to show what I intended it to show, I may have to remove the S font description file from the system.  Further research is required.

Dave <barx>
Project Member
Sun 13 Nov 2022 01:05:57 AM UTC, comment #1: 

Ah, so U+2024 and U+2025 (the one- and two-dot leader characters) aren't intended for modern use; they exist only for compatibility with Xerox Character Code Standard 1980.

http://scripts.sil.org/CharStories_2024

I reckon I don't see any harm in leaving their fallback definitions in tmac/fallbacks.tmac, but I sure wouldn't sweat the details of how they render.

Dave <barx>
Project Member
Sat 12 Nov 2022 11:06:51 PM UTC, original submission:  

A few of the fallback definitions in tmac/fallbacks.tmac could be refined for robustness, typographic quality, or code consistency.


The file organization reflects its assemblage over time: it begins with two blocks of \[u....] characters (one for diacritics, one for roman numerals), goes into some mnemonic Latin-* fallbacks, then goes back to \[u....] definitions (these added in (commit 132182bd, fixing bug #58930).  To me it makes more sense to have all the Unicode definitions together, though perhaps there's some logic to the current arrangement I haven't discerned.

Also, above some of the blocks is a simple comment giving the basic purpose of the block, while others lack this.


As comment 23 of bug #58930 points out in two other cases, the user is likelier to .tr simple ASCII characters than escapes representing those characters.  Most of the Unicode characters defined in the final block do use escapes in their definitions.  Some of those that don't are probably OK because users are unlikely to .tr characters extremely common in text such as the period and the hyphen.  But two definitions might benefit from guarding against this:

  • \[u2016], currently defined as two "|"s, might be more robustly defined as two "\[ba]"s.
  • \[u2052] is defined as "%".  This is trickier to change, because bug #63334 precludes the use of the alternate \[u0025], and groff provides no other way to specify this character.  So we either have to live with this small risk, or make this bug dependent on 63334.

The figure dash (\[u2016]) as currently designed works well with some figures but not others, in at least the TR font.  Consider the PostScript output of:

echo '7\[u2012]5 6\[u2012]0' | groff

In the second pair of figures, the dash touches the figure on either side; ideally there should be a bit of space between the dash and any adjacent figure.  (This will have to amount to shortening the dash, since the space taken up by the entire glyph must remain constant.)


Over in bug #58930 I brought up the spacing between adjacent \[u2026]s.  I quote your reply there (in comment 24):

> I'm tempted to punt on this one, too.  Possibly no serious
> font for typesetting even needs to encode these characters,


If you mean that most fonts will already have a U+2026 glyph, and therefore this fallback will rarely be used, I agree.  (If you mean something else, can you please clarify?)  Every general-purpose font groff ships includes this character.  Further, the Symbol font (which also includes it) being a default special font means that even users using a locally installed font that lacks a U+2026 will get it for free without doing any extra work.

All this might argue for removing this definition from fallbacks.tmac entirely.

> and in groff, if you want a well-typeset leader, a fundamental
> formatter feature (Control+A) will give you one of whatever
> length you like.


True, but a leader is not the only situation where one might use adjacent ellipses.  Someone writing dialogue might decide that "Um\[u2026]\[u2026]" conveys a greater degree of uncertainty than simply "Um\[u2026]".  This will look fine with Times's or Symbol's ellipsis, but the fallbacks.tmac one, should it ever be pressed into service, will not fare so well:

.nf
Baseline demonstration.
\[u2026]\[u2026]\[u2026]
Switch to a font without u2026, and remove the Symbol fallback.
.ft ZD
.special
\[u2026]\[u2026]\[u2026]

I see no problems with the \[u2024] definition, being a single character.  The \[u2025] one will exhibit the problem shown above.  But I don't know in what situations one even uses U+2025, so I'd agree that punting in this case seems the best option.

Dave <barx>
Project Member

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by bjarniig (Posted a comment)
  • -email is unavailable- added by gbranden (Posted a comment)
  • -email is unavailable- added by barx (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

     

     

     

    Follow 4 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2022-12-29 gbranden StatusNone Need Info
    2022-12-28 gbranden Severity3 - Normal 2 - Minor
    2022-12-27 gbranden Severity4 - Important 3 - Normal
    2022-12-26 gbranden Severity2 - Minor 4 - Important

    Back to the top

    Powered by Savane 3.9