bugGNU roff - Bugs: bug #64155, Specifying -fZD on command line...

 
 

bug #64155: Specifying -fZD on command line generates warnings

Submitter:  Dave <barx>
Submitted:  Fri 05 May 2023 11:41:19 PM UTC
   
 
Category:  Macro package - others/general Severity:  3 - Normal
Item Group:  Warning/Suspicious behaviour Status:  Fixed
Privacy:  Public Assigned to:  gbranden
Open/Closed:  Closed Planned Release:  1.24.0
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Sun 19 May 2024 11:50:13 PM UTC, comment #45: 

comment #44:

> I suspect you're not using echo(1) the way you think you are.


I admit I didn't consider its portability, but on my system it output what I intended.

> Because the default style is 'R', and the font 'ZDR' exists (on
> the  "ps" default output device), this actually works.


Yes, I misinterpreted my results earlier.  In fact groff does fail on a bogus family.

$ groff -fBOGUS
troff: fatal error: invalid default font family 'BOGUS'

So there's nothing more to be done here.  Sorry for the red herring.

Dave <barx>
Group Member
Sat 18 May 2024 09:11:43 PM UTC, comment #44: 

comment #43:

> A vast improvement!
>
> But maybe a little too quiet: groff disregards the -fZD flag without telling the user it's doing so.
>
> echo "\N'110'" | groff -ww -fZD
>
> This fails to load the ZD family (because there is no such thing) or the ZD font (because that's not what -f does), so outputs an "n" rather than a solid square.  The invalid -f ought to at least provoke a warning, at least when all warnings are activate.


I can't reproduce this.  I suspect you're not using echo(1) the way you think you are.  Backslash escape sequences are not a portable application of it.  POSIX specifies them, but (a) you need to double the backslash in this case and (b) some environments still won't support them, claiming that the only feature echo should support is an `-n` flag.

https://pubs.opengroup.org/onlinepubs/9699919799/utilities/echo.html

I changed glyphs to a snowflake to make things work more obviously.


$ groff --version | tail -n 1
GNU troff (groff) version 1.23.0.1258-f66cd
$ printf "\\N'100'\n" | groff -ww -fZD >| dave.ps
$ tail dave.ps
def/PL 792 def/LS false def
%%EndSetup
%%Page: 1 1
%%BeginPageSetup
BP
%%EndPageSetup
/F0 10/ZapfDingbats-Reverse SF(d)72 12 Q 0 Cg EP
%%Trailer
end
%%EOF


Because the default style is 'R', and the font 'ZDR' exists (on the  "ps" default output device), this actually works.

If I change to a device that doesn't have 'ZDR', I get an error.


$ printf "\\N'100'\n" | groff -Tdvi -ww -fZD >| dave.ps
troff: fatal error: invalid default font family 'ZD'


G. Branden Robinson <gbranden>
Group administrator
Sat 18 May 2024 08:13:44 PM UTC, comment #43: 

comment #41:

>     Now:
>    
>     $ echo | ./build/test-groff -fZD -a
>     <beginning of page>


A vast improvement!

But maybe a little too quiet: groff disregards the -fZD flag without telling the user it's doing so.

echo "\N'110'" | groff -ww -fZD

This fails to load the ZD family (because there is no such thing) or the ZD font (because that's not what -f does), so outputs an "n" rather than a solid square.  The invalid -f ought to at least provoke a warning, at least when all warnings are activate.

This could reasonably be considered a separate bug, so I'm happy to open a new report if warranted.

Dave <barx>
Group Member
Tue 14 May 2024 04:37:20 PM UTC, comment #42: 

So here's what got lost last week.

I did in fact implement a string comparison conditional operator in GNU troff in my working copy.

And it worked.


$ git stash show -p 1
diff --git a/src/roff/troff/input.cpp b/src/roff/troff/input.cpp
index 8d828a01e..7288f31c5 100644
--- a/src/roff/troff/input.cpp
+++ b/src/roff/troff/input.cpp
@@ -6002,6 +6002,25 @@ static bool do_if_request()
     }
     result = is_abstract_style(nm);
   }
+  else if (c == '~') {
+    tok.next();
+    symbol nm1 = get_name(true /* required */);
+    symbol nm2 = get_name(true /* required */);
+    if ((nm1.is_null() || nm2.is_null())) {
+      skip_branch();
+      return false;
+    }
+    macro *rm1 = static_cast<macro *>(request_dictionary.lookup(nm1));
+    macro *rm2 = static_cast<macro *>(request_dictionary.lookup(nm2));
+    macro *m1 = rm1->to_macro();
+    macro *m2 = rm2->to_macro();
+    if ((0 /* nullptr */ == m1) || (0 /* nullptr */ == m2)) {
+      error("cannot perform equality comparison on a request");
+      skip_branch();
+      return false;
+    }
+    result = (*m1 == *m2);
+  }
   else if (tok.is_space())
     result = false;
   else if (tok.is_usable_as_delimiter()) {


But I immediately thought better of it.

1.  We already have


.if '\?foo\?'\?bar\?' .tm strings are equal


2.  The `\?` escape sequence is already a GNU troff extension.

3.  I didn't feel good about adding syntactic sugar for this form of expression.

4.  String comparisons are rare outside of macro files.

5.  Macro files desiring to do string comparisons should decide whether they'd rather let them be formatted (a popular choice, even in groff's own macro files) or not.  In most cases there is no harm in performing formatting on the comparands.  (Where there is, you must use `\?`.)

...

This is growing long.  I will return to it later.

G. Branden Robinson <gbranden>
Group administrator
Tue 14 May 2024 01:24:26 PM UTC, comment #41: 

I'll follow up with a brief (I hope) recapitulation of my lost comment.

The following bug-closing context will help reduce the length of that.


commit 609d962fc02e517744ee4f68160648a78df3a020
Author: G. Branden Robinson <g.branden.robinson@gmail.com>
Date:   Sat May 11 09:54:35 2024 -0500

    [tmac]: Perform actual string comparisons.

    * tmac/fallbacks.tmac:
    * tmac/troffrc-end: Bracket comparands to formatted output comparison
      operator with `\?` escape sequences to perform a more elementary
      string comparison on them (cf. a comparison of _formatted text_).
      This way they continue to perform their function even if the default
      font family has no coverage of Basic Latin (not a bizarre choice when
      considering potential rendering of documents using Devanagari or East
      Asian languages).  Unfortunately this means of comparing strings,
      while long documented in the GNU troff Manual, is neither idiomatic
      for *roff macro programmers nor portable to AT&T troff.  (The *roff
      language historically _lacks_ a string comparison operator.)  Each
      macro package will have to decide for itself whether it wants to have
      portability at the expense of requiring Basic Latin coverage (possibly
      switching fonts before and after a formatted output comparison for the
      sole purpose of obtaining it).  These two files are unconditionally
      loaded by the stock `troffrc`, and therefore must be dealt with in any
      case.

    Fixes <https://savannah.gnu.org/bugs/?64155>.  Thanks to Dave Kemper for
    the report, and to him, Peter Schaffter, and Deri James for the
    discussion.

    Previously:

    $ echo | groff -fZD -a
    troff:/.../groff/1.23.0/tmac/fallbacks.tmac:14: warning: character 'p' not defined
    troff:/.../groff/1.23.0/tmac/fallbacks.tmac:14: warning: character 's' not defined
    troff:/.../groff/1.23.0/tmac/fallbacks.tmac:14: warning: character 'a' not defined
    troff:/.../groff/1.23.0/tmac/fallbacks.tmac:14: warning: character 'c' not defined
    troff:/.../groff/1.23.0/tmac/fallbacks.tmac:14: warning: character 'i' not defined
    troff:/.../groff/1.23.0/tmac/fallbacks.tmac:16: warning: character 'l' not defined
    troff:/.../groff/1.23.0/tmac/fallbacks.tmac:16: warning: character 't' not defined
    troff:/.../groff/1.23.0/tmac/fallbacks.tmac:16: warning: character 'n' not defined
    troff:/.../groff/1.23.0/tmac/fallbacks.tmac:17: warning: character 'u' not defined
    troff:/.../groff/1.23.0/tmac/fallbacks.tmac:17: warning: character 'f' not defined
    troff:/.../groff/1.23.0/tmac/troffrc-end:7: warning: character 'h' not defined
    troff:/.../groff/1.23.0/tmac/troffrc-end:7: warning: character 'm' not defined
    <beginning of page>

    Now:

    $ echo | ./build/test-groff -fZD -a
    <beginning of page>


G. Branden Robinson <gbranden>
Group administrator
Sat 11 May 2024 02:56:39 PM UTC, comment #40: 

I have a fix for this involving something old and in plain sight.

G. Branden Robinson <gbranden>
Group administrator
Wed 08 May 2024 09:55:07 PM UTC, comment #39: 

Self-followup:

comment #37:

> in a way back compatible to almost any older troff.


Obviously, this specific example is not AT&T-portable due to the long names, but the logic can be written portably.

Dave <barx>
Group Member
Mon 06 May 2024 04:17:11 AM UTC, comment #38: 
G. Branden Robinson <gbranden>
Group administrator
Mon 06 May 2024 04:10:59 AM UTC, comment #37: 

comment #34:

> Maybe it is time for a proper string equality operator in our
> conditional expressions.


While that would be a useful addition to the language, equivalent functionality can be achieved in user space, in a way back compatible to almost any older troff.

.de string_compare
.  ft TR
.  ie '\\$1'\\$2' .nr strings_equal 1
.  el .nr strings_equal 0
.  ft P
..
.
.ft ZD
.
.if 'foo'foo' .tm foo = foo, via output comparison operator
.if 'foo'bar' .tm foo = bar, via output comparison operator
.
.string_compare foo foo
.if \n[strings_equal] .tm foo = foo, via string_compare macro
.string_compare foo bar
.if \n[strings_equal] .tm foo = bar, via string_compare macro


Dave <barx>
Group Member
Mon 06 May 2024 04:02:45 AM UTC, comment #36: 

Sigh.  Wrote a long comment.  Form was stale.  Document Expired.  Text lost.

G. Branden Robinson <gbranden>
Group administrator
Fri 03 May 2024 04:04:32 PM UTC, comment #35: 

comment #34:

> > > Well, when you're prepared to discuss it, it would be good to know if/how Dave's original report in comment #0 was invalid
> >
> > I think I can answer this - it is certainly not invalid, it just has nothing to do with ZD not being a "proper" family (in your eyes).
>
> This isn't a matter of my subjective opinion; the "family" and "style" features of groff date back to 1990.
>
> https://git.savannah.gnu.org/cgit/groff.git/tree/ChangeLog.115?h=1.23.0#n5535
>
> There is a long AT&T troff hangover of assuming that `R`, `I`, and `B` fonts will exist (or, in groff, that abstract styles that work analogously to fonts of those names will).  This assumption can extend to `S` as well.


I don't dispute R, I, B, and BI is a family, it's not the only possible family, previously a font family of just R and B would be happily accepted and \fBword\fP worked as expected. It was your code which insisted on a far more inflexible rule.

[...]

> > If you do -fZD, which has no alphabetic glyphs, any start up macros which contain conditional statements of the form:-
> >
> > .if '\*[.T]'html' \" etc..
> >
> > Will produce character not found errors, since, as we know, both sides are formatted in separate environments and compared.
>
> Yes.  I find this troublesome.
>
> It seems to be an idiom to use the output comparison operator (a term I had to coin, for it had no name) for string comparisons even when there is no intention of formatting either of the comparands as text.
>
> But that idiom would seem to port poorly to document environments where the basic Latin alphabet doesn't need to be formatted at all.
>
> > The give away in Dave's initial report is that the "character undefined" errors spell out the words "ps/pdf/html" at least can be seen.
>
> You paid better attention than I did.  I managed to overlook it, staring more closely at the 1.23.0 output, seeing only the character sequence "psaciltnufhm", and deducing nothing much in particular from that.
>
> Good sleuthing!
>
> But also there's no way in hell we should be spewing diagnostics on, or failing to correctly perform comparisons within, control structures that are initializing the state of the formatter.  These macro files are profoundly uninterested in formatting any output, and they'd be buggy if they did so, since an empty input document should produce no output no matter what (stock) macro packages or startup macro files you load.
>
> Maybe it is time for a proper string equality operator in our conditional expressions.


Ooh, yes please.

> > I'm sure this "feature" has come up before. Even with Branden's code which stopped -fZD from working did not address the real problem because if you have four fonts with R, I, B, and BI extension but don't contain alphabetic characters, -f works for the "family" but you will see exactly the same errors as Dave reported.
>
> Fair point.

> > One way for a proper fix, is to create a copy of TR as TRSKEL, add the "special" parameter, and change the name parameter to TRSKEL, remove all kernpairs, delete all glyph definitions above 127, and finally alter the DESC file by incrementing the number and adding TRSKEL on the end. This will solve the error occurring if you use -f on fonts which don't contain ascii glyphs, such as some CJK fonts. This can all be done in the devpdf directory (I have done it for testing), but a very similar change can be made to devps as well.
>
> Here, I must disagree.  The foregoing strikes me as a workaround rather than a proper fix.


I agree, it's a workaround, but it is also a proper fix in the sense that it prevented Dave's errors appearing for non-latin fonts, which you agreed your code did not (so was not a proper fix).

[...]

> A string equality test.  Not a comparison of node lists, which are laden with all sorts of other data.[1]  Just a string equality test.
>
> Well, let's give the people what they are asking for.  Especially since a lot of the time, "the people" is us.


+1 here, that would be a proper fix too. What semantics are you considering for this new conditional?

> [1] Now that I've started resurrecting and expanding the formatter's facilities for node dumping, I'm getting a better notion of what all that extra data is.  I begin to revise upward my expectations of the performance improvement I can expect from a straight-up simple string equality operator, particularly since the operation can be handed off to the C++ standard library and from there, likely to platform-specific, optimized assembly code.  It hasn't escaped me that this might also help, say, string comparisons of PDF bookmark tags, since they too are not output to be formatted.


I liked .pline.

Deri James <deri>
Group Member
Fri 03 May 2024 06:01:48 AM UTC, comment #34: 


commit 604361951c9ddad621b3e0786ca7a2946e212842
Author: G. Branden Robinson <g.branden.robinson@gmail.com>
Date:   Fri May 3 00:00:59 2024 -0500

    Revert "[troff]: Validate a font family before using it."

    This reverts commit 39ffa368dc6a1de4c11cf3f4f5b8594d3c974173.

    There were a few problems with this approach; possibly the biggest is
    that styles can be dynamically constructed, for which this validation
    process was no help, and could be a misleading hindrance.

    Thanks to Dave Kemper, Deri James, and Peter Schaffter for the
    discussion.

    <https://savannah.gnu.org/bugs/?64155> is already reopened.


comment #29:

> > Well, when you're prepared to discuss it, it would be good to know if/how Dave's original report in comment #0 was invalid
>
> I think I can answer this - it is certainly not invalid, it just has nothing to do with ZD not being a "proper" family (in your eyes).


This isn't a matter of my subjective opinion; the "family" and "style" features of groff date back to 1990.

https://git.savannah.gnu.org/cgit/groff.git/tree/ChangeLog.115?h=1.23.0#n5535

There is a long AT&T troff hangover of assuming that `R`, `I`, and `B` fonts will exist (or, in groff, that abstract styles that work analogously to fonts of those names will).  This assumption can extend to `S` as well.


$ git blame font/devhtml/DESC.proto
fe93746e79 (Werner LEMBERG 2001-01-17 14:53:37 +0000  1) res 240
fe93746e79 (Werner LEMBERG 2001-01-17 14:53:37 +0000  2) hor 24
fe93746e79 (Werner LEMBERG 2001-01-17 14:53:37 +0000  3) vert 40
fe93746e79 (Werner LEMBERG 2001-01-17 14:53:37 +0000  4) unitwidth 10
6d5bbb4fbb (Werner LEMBERG 2003-03-31 14:31:21 +0000  5) sizes 1-1000 0
2738e8ceb6 (Werner LEMBERG 2002-08-07 15:01:32 +0000  6) fonts 9 R I B BI CR CI CB CBI S
fe93746e79 (Werner LEMBERG 2001-01-17 14:53:37 +0000  7) tcommand
8fe7c747b7 (Werner LEMBERG 2004-10-08 07:08:08 +0000  8) unscaled_charwidths
fe93746e79 (Werner LEMBERG 2001-01-17 14:53:37 +0000  9) postpro post-grohtml
fe93746e79 (Werner LEMBERG 2001-01-17 14:53:37 +0000 10) prepro  pre-grohtml
fe93746e79 (Werner LEMBERG 2001-01-17 14:53:37 +0000 11) use_charnames_in_special
fe93746e79 (Werner LEMBERG 2001-01-17 14:53:37 +0000 12) pass_filenames
4d917850b2 (Werner LEMBERG 2006-02-26 22:21:38 +0000 13) unicode


Eventually, I documented the foregoing.

Your critique implies that we should not further entrench assumptions regarding the availability of font or style names.  Fair enough.

> If you do -fZD, which has no alphabetic glyphs, any start up macros which contain conditional statements of the form:-
>
> .if '\*[.T]'html' \" etc..
>
> Will produce character not found errors, since, as we know, both sides are formatted in separate environments and compared.


Yes.  I find this troublesome.

It seems to be an idiom to use the output comparison operator (a term I had to coin, for it had no name) for string comparisons even when there is no intention of formatting either of the comparands as text.

But that idiom would seem to port poorly to document environments where the basic Latin alphabet doesn't need to be formatted at all.

> The give away in Dave's initial report is that the "character undefined" errors spell out the words "ps/pdf/html" at least can be seen.


You paid better attention than I did.  I managed to overlook it, staring more closely at the 1.23.0 output, seeing only the character sequence "psaciltnufhm", and deducing nothing much in particular from that.

Good sleuthing!

But also there's no way in hell we should be spewing diagnostics on, or failing to correctly perform comparisons within, control structures that are initializing the state of the formatter.  These macro files are profoundly uninterested in formatting any output, and they'd be buggy if they did so, since an empty input document should produce no output no matter what (stock) macro packages or startup macro files you load.

Maybe it is time for a proper string equality operator in our conditional expressions.

> I'm sure this "feature" has come up before. Even with Branden's code which stopped -fZD from working did not address the real problem because if you have four fonts with R, I, B, and BI extension but don't contain alphabetic characters, -f works for the "family" but you will see exactly the same errors as Dave reported.


Fair point.
 

> One way for a proper fix, is to create a copy of TR as TRSKEL, add the "special" parameter, and change the name parameter to TRSKEL, remove all kernpairs, delete all glyph definitions above 127, and finally alter the DESC file by incrementing the number and adding TRSKEL on the end. This will solve the error occurring if you use -f on fonts which don't contain ascii glyphs, such as some CJK fonts. This can all be done in the devpdf directory (I have done it for testing), but a very similar change can be made to devps as well.


Here, I must disagree.  The foregoing strikes me as a workaround rather than a proper fix.

> An alternative fix would be to consider including the GNU UnifontMedium font in groff and using it as a special instead of TRSKEL, this has the advantage of solving any undefined glyphs as well, but there are other issues.


I'm not thrilled with this one either.  A document of Chinese poetry or a setting of the Rig Veda need never format any Basic Latin glyphs.  At the same time, it might want to issue conditional requests predicated on the identity of the output device.  That's a reasonable thing to want.  Why drag in a font to get that functionality?

When we look at the macro files that produced the diagnostic messages when the `-fZD` option was given, we should ask ourselves (as you did): what is the macro file trying to do that generates all this noise?

A string equality test.  Not a comparison of node lists, which are laden with all sorts of other data.[1]  Just a string equality test.

Well, let's give the people what they are asking for.  Especially since a lot of the time, "the people" is us.

[1] Now that I've started resurrecting and expanding the formatter's facilities for node dumping, I'm getting a better notion of what all that extra data is.  I begin to revise upward my expectations of the performance improvement I can expect from a straight-up simple string equality operator, particularly since the operation can be handed off to the C++ standard library and from there, likely to platform-specific, optimized assembly code.  It hasn't escaped me that this might also help, say, string comparisons of PDF bookmark tags, since they too are not output to be formatted.

G. Branden Robinson <gbranden>
Group administrator
Fri 03 May 2024 04:43:19 AM UTC, comment #33: 

comment #32:

> comment #28:
> > comment #27:
> > Specifically, am I correct to claim either of the following?
> >
> > A.  "[G]iven that mom has her own system of managing fonts, and part of her contract with the user [...] is that [the] user will not go behind her back and start invoking *roff requests." is a false statement.  (Possibly an exaggeration.)
>
> Oversimplification, possibly my fault.  You got the idea from this bit of the documentation:
>
> "In some cases, mom’s typesetting macros merely imitate groff primitives. In others, they approach typesetting concerns in conceptually new ways (for groff, at least). This should present no problem for newcomers to groff who are learning mom. Old groff hands should be careful. Just because it looks like a duck and walks like a duck does not, in this instance, mean that it is a duck. When using mom, stay away from groff primitives if mom provides a macro that accomplishes the same thing."


Yes.  I haven't read anything like all of mom's documentation, but the foregoing sounds a fairly large bell in my head.  I likely did overinterpret it.

> That's not a contract, it's a recommendation.  I don't want users imagining. for example, that they can use either .ps or .PT_SIZE interchangeably (or .ft/.FT) and expect the same results.  E.g. if AUTOLEAD is enabled, .PT_SIZE changes the pointsize and updates the leading.  Plain .ps only changes the size, hence the recommendation to stay away from groff primitives if mom provides a macro that accomplishes the same thing.  There a number of macros where the documentation explicitly states that using a primitive instead of a macro is fine.


Acknowledged.
 

> I'm not comfortable with the statement "mom has her own system of managing fonts."  Other than that .FAM and .FT perform checks and set registers needed by other macros, and the inclusion of pre-defined supplementary styles (none loaded in positions 1-4), there is nothing unique about mom's font management.


No indeed; given your clarification above, item A is greatly overstated.

> > B.  The statement "By issuing appropriate formatter instructions, you can override these defaults before your document writes its first glyph." in our manual should be dropped, or revised to stipulate that some macro packages (namely mom), will assume that that before a document requests a glyph to be formatted, mounting position 1 will be assigned to a style named 'R'.
>
> I'm confused.  The docs currently say, "The default mounting position, and therefore style, is always '1' ('R')"  Why would this suddenly only apply to "some" macro packages?  I don't think the B. statement should be dropped, but I would change "these defaults" because the only formatter flag pertinent to that section of the documentation is -f <family>.


I think I can probably leave this item as-is.  I certainly should not add any special caveat about mom, but I think the "these defaults" sentence can stay once the larger context, an introduction to groff's font concept, is considered.

Here is that context.


5.19 Using Fonts
================

In digital typography, a "font" is a collection of characters in a
specific typeface that a device can render as glyphs at a desired
size.(1)  (*note Using Fonts-Footnote-1::) A 'roff' formatter can change
typefaces at any point in the text.  The basic faces are a set of
"styles" combining upright and slanted shapes with normal and heavy
stroke weights: 'R', 'I', 'B', and 'BI'--these stand for roman, italic,
bold, and bold-italic.  For linguistic text, GNU 'troff' groups
typefaces into "families" containing each of these styles.(2)  (*note
Using Fonts-Footnote-2::) A "text font" is thus often a family combined
with a style, but it need not be: consider the 'ps' and 'pdf' devices'
'ZCMI' (Zapf Chancery Medium italic)--often, no other style of Zapf
Chancery Medium is provided.  On typesetters, at least one "special
font" is available, comprising "unstyled" glyphs for mathematical
operators and other purposes.

   Like the AT&T 'troff' formatter, GNU 'troff' does not itself load or
manipulate a digital font file;(3) (*note Using Fonts-Footnote-3::)
instead it works with a "font description file" that characterizes it,
including its glyph repertoire and the "metrics" (dimensions) of each
glyph.(4)  (*note Using Fonts-Footnote-4::) This information permits the
formatter to accurately place glyphs with respect to each other.  Before
using a font description, the formatter associates it with a "mounting
position", a place in an ordered list of available typefaces.  So that a
document need not be strongly coupled to a specific font family, in GNU
'troff' an output device can associate a style in the abstract sense
with a mounting position.  Thus the default family can be combined with
a style dynamically, producing a "resolved font name".  A user-specified
font name that combines family and style (or refers to a font that is
not a member of a family) is already "resolved".

   Fonts often have trademarked names, and even Free Software fonts can
require renaming upon modification.  'groff' maintains a convention that
a device's serif font family is given the name 'T' ("Times"), its
sans-serif family 'H' ("Helvetica"), and its monospaced family 'C'
("Courier").  Historical inertia has driven 'groff''s font identifiers
to short uppercase abbreviations of font names, as with 'TR', 'TI',
'TB', 'TBI', and a special font 'S'.

   The default family used with abstract styles is initially 'T'.
Typically, abstract styles are arranged in the first four mounting
positions in the order shown above.  The default mounting position, and
therefore style, is always '1' ('R').  By issuing appropriate formatter
instructions, you can override these defaults before your document
writes its first glyph.

   Terminals cannot change font families and lack special fonts.  They
support style changes by overstriking, or by altering ISO 6429/ECMA-48
"graphic renditions" (character cell attributes).


I've set the status back to "Confirmed" because my way forward--for the time being--seems clear.

I think Dave's point in comment #31 is worthwhile and demands further consideration.  I don't relish the labor or I/O expense of making a `fam` request hit the file system repeatedly, speculatively loading every registered style for the family being switched to, but maybe that is what is warranted.

G. Branden Robinson <gbranden>
Group administrator
Fri 03 May 2024 01:46:14 AM UTC, comment #32: 


comment #28:

> comment #27:
> > Hi Peter,
>
> I should have spent a little longer on that comment since I tossed the ball back into your court, even if just for advice.
>
> > Please confirm my understanding of the foregoing and I will proceed with the reversion right away.
>
> Specifically, am I correct to claim either of the following?
>
> A.  "[G]iven that mom has her own system of managing fonts, and part of her contract with the user [...] is that [the] user will not go behind her back and start invoking *roff requests." is a false statement.  (Possibly an exaggeration.)


Oversimplification, possibly my fault.  You got the idea from this bit of the documentation:

"In some cases, mom’s typesetting macros merely imitate groff primitives. In others, they approach typesetting concerns in conceptually new ways (for groff, at least). This should present no problem for newcomers to groff who are learning mom. Old groff hands should be careful. Just because it looks like a duck and walks like a duck does not, in this instance, mean that it is a duck. When using mom, stay away from groff primitives if mom provides a macro that accomplishes the same thing."

That's not a contract, it's a recommendation.  I don't want users imagining. for example, that they can use either .ps or .PT_SIZE interchangeably (or .ft/.FT) and expect the same results.  E.g. if AUTOLEAD is enabled, .PT_SIZE changes the pointsize and updates the leading.  Plain .ps only changes the size, hence the recommendation to stay away from groff primitives if mom provides a macro that accomplishes the same thing.  There a number of macros where the documentation explicitly states that using a primitive instead of a macro is fine.

I'm not comfortable with the statement "mom has her own system of managing fonts."  Other than that .FAM and .FT perform checks and set registers needed by other macros, and the inclusion of pre-defined supplementary styles (none loaded in positions 1-4), there is nothing unique about mom's font management.
 

> B.  The statement "By issuing appropriate formatter instructions, you can override these defaults before your document writes its first glyph." in our manual should be dropped, or revised to stipulate that some macro packages (namely mom), will assume that that before a document requests a glyph to be formatted, mounting position 1 will be assigned to a style named 'R'.


I'm confused.  The docs currently say, "The default mounting position, and therefore style, is always '1' ('R')"  Why would this suddenly only apply to "some" macro packages?  I don't think the B. statement should be dropped, but I would change "these defaults" because the only formatter flag pertinent to that section of the documentation is -f <family>.

Peter Schaffter <PTPi>
Group Member
Fri 03 May 2024 01:34:51 AM UTC, comment #31: 

comment #27:

> it would be good to know if/how Dave's original report in
> comment #0 was invalid.


I'm fine with -fZD failing on the grounds that groff doesn't consider ZD a family, but the failure mechanism should be cleaner.

Dave <barx>
Group Member
Fri 03 May 2024 12:24:39 AM UTC, comment #30: 

Thanks, Deri--that analysis was highly illuminating.  I feel I know what I need now to write a rationale committing the reversion.

That will take a little time, as will composing a proper response to comment #29 where I expect to mull alternative avenues for solving the problem of comment #0, but y'all can expect the reversion to land overnight UTC.

G. Branden Robinson <gbranden>
Group administrator
Fri 03 May 2024 12:00:43 AM UTC, comment #29: 


> Well, when you're prepared to discuss it, it would be good to know if/how Dave's original report in comment #0 was invalid


I think I can answer this - it is certainly not invalid, it just has nothing to do with ZD not being a "proper" family (in your eyes). If you do -fZD, which has no alphabetic glyphs, any start up macros which contain conditional statements of the form:-

.if '\*[.T]'html' \" etc..

Will produce character not found errors, since, as we know, both sides are formatted in separate environments and compared.

The give away in Dave's initial report is that the "character undefined" errors spell out the words "ps/pdf/html" at least can be seen. I'm sure this "feature" has come up before. Even with Branden's code which stopped -fZD from working did not address the real problem because if you have four fonts with R, I, B, and BI extension but don't contain alphabetic characters, -f works for the "family" but you will see exactly the same errors as Dave reported.

One way for a proper fix, is to create a copy of TR as TRSKEL, add the "special" parameter, and change the name parameter to TRSKEL, remove all kernpairs, delete all glyph definitions above 127, and finally alter the DESC file by incrementing the number and adding TRSKEL on the end. This will solve the error occurring if you use -f on fonts which don't contain ascii glyphs, such as some CJK fonts. This can all be done in the devpdf directory (I have done it for testing), but a very similar change can be made to devps as well.

An alternative fix would be to consider including the GNU UnifontMedium font in groff and using it as a special instead of TRSKEL, this has the advantage of solving any undefined glyphs as well, but there are other issues.

Deri James <deri>
Group Member
Thu 02 May 2024 10:30:39 PM UTC, comment #28: 

comment #27:

> Hi Peter,


I should have spent a little longer on that comment since I tossed the ball back into your court, even if just for advice.

> Please confirm my understanding of the foregoing and I will proceed with the reversion right away.


Specifically, am I correct to claim either of the following?

A.  "[G]iven that mom has her own system of managing fonts, and part of her contract with the user [...] is that [the] user will not go behind her back and start invoking *roff requests." is a false statement.  (Possibly an exaggeration.)

B.  The statement "By issuing appropriate formatter instructions, you can override these defaults before your document writes its first glyph." in our manual should be dropped, or revised to stipulate that some macro packages (namely mom), will assume that that before a document requests a glyph to be formatted, mounting position 1 will be assigned to a style named 'R'.

Thanks for any light you can shed here.

G. Branden Robinson <gbranden>
Group administrator
Thu 02 May 2024 10:17:05 PM UTC, comment #27: 

Hi Peter,

comment #26:

> I guess I wasn't clear enough when I advised not implementing .fam checking as proposed by Branden,


Please be reassured that this wasn't a matter of proceeding over your protest (5 March 2024); the commit had already been in place on the master branch since 10 July 2023.
 

> as I have just discovered the change has been committed, with the expected consequence: partial families in my library that have only R and I fonts choke on .fam.
>
>   printf ".fam Technical\n.ft R" | groff -z
>   troff:<standard input>:1: error: 'Technical' is not a valid font family
>
> This is a regression.


This example invalidates one of the premises I put forward in comment #22 ("given that mom has her own system of managing fonts, and part of her contract with the user, as I understand it, is that [the] user will not go behind her back and start invoking *roff requests") and regarding which I solicited your input.

It may also invalidate some language I put into our Texinfo manual.


   The default family used with abstract styles is initially 'T'.
Typically, abstract styles are arranged in the first four mounting
positions in the order shown above.  The default mounting position, and
therefore style, is always '1' ('R').  By issuing appropriate formatter
instructions, you can override these defaults before your document
writes its first glyph.


...namely, the final sentence.

Please confirm my understanding of the foregoing and I will proceed with the reversion right away.

> Please revert the commit that introduced it,
> 39ffa368dc6a1de4c11cf3f4f5b8594d3c974173.  I don't see any need for .fam checking in the first place, but that discussion should be left for after the commit is reverted.


Well, when you're prepared to discuss it, it would be good to know if/how Dave's original report in comment #0 was invalid.

G. Branden Robinson <gbranden>
Group administrator
Thu 02 May 2024 08:52:36 PM UTC, comment #26: 

I guess I wasn't clear enough when I advised not implementing .fam checking as proposed by Branden, as I have just discovered the change has been committed, with the expected consequence: partial families in my library that have only R and I fonts choke on .fam.

  printf ".fam Technical\n.ft R" | groff -z
  troff:<standard input>:1: error: 'Technical' is not a valid font family

This is a regression.  Please revert the commit that introduced it,
39ffa368dc6a1de4c11cf3f4f5b8594d3c974173.  I don't see any need for .fam checking in the first place, but that discussion should be left for after the commit is reverted.

Peter Schaffter <PTPi>
Group Member
Wed 06 Mar 2024 08:02:31 PM UTC, comment #25: 

Thanks, Peter.  In light of comment #24, taking this out of Need Info status and reassigning to Branden.

Dave <barx>
Group Member
Tue 05 Mar 2024 04:03:02 AM UTC, comment #24: 

The restriction does conflict with mom's supplementary styles, however  the conflict is not unique to mom.  Any user-installed family may not have all of R/I/B/BI and, if a user gets creative with .sty, may not, in fact, have any of them.  I have dozens of partial families, and plenty with just .sty's taken from mom's offerings

Peter Schaffter <PTPi>
Group Member
Tue 05 Mar 2024 01:04:59 AM UTC, comment #23: 

comment #21:

> this code:-

...

> Copied the corrupt file rather than the correct file in
> ./font/devpdf. What is the purpose of this?


This issue being unrelated to this ticket's original purpose (which was fixed and closed with comment #17), I opened bug #65415 to address it separately.

> I am going to leave this open so that you can reply to

...

> whether you think this change will affect mom's use of .fam.


This seems to be the only other question keeping this ticket open now.  Ping: Peter.  (Also assigning the bug to him for now, since Deri has withdrawn from gropdf work.)

Dave <barx>
Group Member
Tue 06 Feb 2024 01:23:36 PM UTC, comment #22: 

comment #21:

> Hi Branden,
>
> I've worked it out. This is the problem, I had a corrupt copy of U-TR in my build directory


Ok.  That explains the different file size as noted in comment #20.

> so this code:-


>   if test -f $f; then \
>     /usr/bin/install -c -m 644 $f /usr/share/groff/1.23.0/font/devpdf/$f; \
>   else \
>     /usr/bin/install -c -m 644 ./font/devpdf/$f \
>       /usr/share/groff/1.23.0/font/devpdf/$f; \
>   fi; \


> Copied the corrupt file rather than the correct file in ./font/devpdf. What is the purpose of this?


I don't know.  My name is on the 2 lines of the else branch per "git blame" but closer inspection reveals that all I did was break the long lines.  Before that was a commit by Bertrand.

https://git.savannah.gnu.org/cgit/groff.git/commit/?id=b101574cae1b3019d4109d72b81e4c0a33bb5a86

and before that it was a "Makefile.sub" written by...

...you.  ;-)

> I am going to leave this open so that you can reply to the above


I'm going to plead noninvolvement on this one.  :P

> and whether you think this change will affect mom's use of .fam.


I don't think so, given that mom has her own system of managing fonts, and part of her contract with the user, as I understand it, is that user will not go behind her back and start invoking *roff requests.

But I'll add Peter to the CC so he can opine.  I trust he'll know if I'm confounding anything mom does.

> Also, I am a bit surprised that you must be reading U-TR (rather than just checking for its existence) and then when it fails (silently) it compains about the family, which is confusing, rather than reporting a font as corrupt.


I agree.  I'm not happy with that, either.  See the end of comment #15.

G. Branden Robinson <gbranden>
Group administrator
Tue 06 Feb 2024 01:04:06 PM UTC, comment #21: 

Hi Branden,

I've worked it out. This is the problem, I had a corrupt copy of U-TR in my build directory so this code:-

  if test -f $f; then \
    /usr/bin/install -c -m 644 $f /usr/share/groff/1.23.0/font/devpdf/$f; \
  else \
    /usr/bin/install -c -m 644 ./font/devpdf/$f \
      /usr/share/groff/1.23.0/font/devpdf/$f; \
  fi; \

Copied the corrupt file rather than the correct file in ./font/devpdf. What is the purpose of this?

I am going to leave this open so that you can reply to the above and whether you think this change will affect mom's use of .fam.

Also, I am a bit surprised that you must be reading U-TR (rather than just checking for its existence) and then when it fails (silently) it compains about the family, which is confusing, rather than reporting a font as corrupt.


Deri James <deri>
Group Member
Tue 06 Feb 2024 01:30:25 AM UTC, comment #20: 

Hi Deri,

comment #19:

> [derij@pip build (master)]$ ll ~/groff-git/groff/build/font/devpdf/U-T*
> -rw-r--r-- 1 derij derij 26563 Feb  5 23:40 /home/derij/groff-git/groff/build/font/devpdf/U-TB
> -rw-r--r-- 1 derij derij 30421 Feb  5 23:40 /home/derij/groff-git/groff/build/font/devpdf/U-TBI
> -rw-r--r-- 1 derij derij 30662 Feb  5 23:40 /home/derij/groff-git/groff/build/font/devpdf/U-TI
> -rw-r--r-- 1 derij derij 26596 Feb  5 23:40 /home/derij/groff-git/groff/build/font/devpdf/U-TR

> [derij@pip build (master)]$ ll /usr/share/groff/1.23.0/font/devpdf/U-T*
> -rw-r--r-- 1 root root 26563 Feb  5 23:44 /usr/share/groff/1.23.0/font/devpdf/U-TB
> -rw-r--r-- 1 root root 30421 Feb  5 23:44 /usr/share/groff/1.23.0/font/devpdf/U-TBI
> -rw-r--r-- 1 root root 30662 Feb  5 23:44 /usr/share/groff/1.23.0/font/devpdf/U-TI
> -rw-r--r-- 1 root root 17000 Feb  5 23:44 /usr/share/groff/1.23.0/font/devpdf/U-TR


Before we go down a rabbit hole of instrumentation and/or GDB backtracing, I noticed something there.

These file sizes are the same except for U-TR.  What's going on there?  Is the U-TR in /usr/share/groff/1.23.0/font/devpdf/ a valid font file?  If so, how does it differ from the one in /home/derij/groff-git/groff/build/font/devpdf/?

G. Branden Robinson <gbranden>
Group administrator
Tue 06 Feb 2024 12:03:47 AM UTC, comment #19: 

Further testing shows that test-groff works but after a make install it fails!!

test-groff

[pid 3056312] openat(AT_FDCWD, "/home/derij/groff-git/groff/build/font/devpdf/U-TR", O_RDONLY) = 3
[pid 3056312] newfstatat(3, "", {st_mode=S_IFREG|0644, st_size=26596, ...}, AT_EMPTY_PATH) = 0
[pid 3056312] openat(AT_FDCWD, "/home/derij/groff-git/groff/build/font/devpdf/U-TI", O_RDONLY) = 3
[pid 3056312] newfstatat(3, "", {st_mode=S_IFREG|0644, st_size=30662, ...}, AT_EMPTY_PATH) = 0
[pid 3056312] openat(AT_FDCWD, "/home/derij/groff-git/groff/build/font/devpdf/U-TB", O_RDONLY) = 3
[pid 3056312] newfstatat(3, "", {st_mode=S_IFREG|0644, st_size=26563, ...}, AT_EMPTY_PATH) = 0
[pid 3056312] openat(AT_FDCWD, "/home/derij/groff-git/groff/build/font/devpdf/U-TBI", O_RDONLY) = 3
[pid 3056312] newfstatat(3, "", {st_mode=S_IFREG|0644, st_size=30421, ...}, AT_EMPTY_PATH) = 0
[pid 3056312] openat(AT_FDCWD, "/etc/localtime", O_RDONLY|O_CLOEXEC) = 3
[pid 3056312] newfstatat(3, "", {st_mode=S_IFREG|0644, st_size=3664, ...}, AT_EMPTY_PATH) = 0
[pid 3056312] newfstatat(3, "", {st_mode=S_IFREG|0644, st_size=3664, ...}, AT_EMPTY_PATH) = 0
[pid 3056312] openat(AT_FDCWD, "/home/derij/groff-git/groff/build/font/devpdf/U-TR", O_RDONLY) = 3

[derij@pip build (master)]$ ll ~/groff-git/groff/build/font/devpdf/U-T*
-rw-r--r-- 1 derij derij 26563 Feb  5 23:40 /home/derij/groff-git/groff/build/font/devpdf/U-TB
-rw-r--r-- 1 derij derij 30421 Feb  5 23:40 /home/derij/groff-git/groff/build/font/devpdf/U-TBI
-rw-r--r-- 1 derij derij 30662 Feb  5 23:40 /home/derij/groff-git/groff/build/font/devpdf/U-TI
-rw-r--r-- 1 derij derij 26596 Feb  5 23:40 /home/derij/groff-git/groff/build/font/devpdf/U-TR

after install (groff)

[pid 3048720] openat(AT_FDCWD, "/usr/share/groff/1.23.0/font/devpdf/U-TR", O_RDONLY) = 3
[pid 3048720] newfstatat(3, "", {st_mode=S_IFREG|0644, st_size=17000, ...}, AT_EMPTY_PATH) = 0
troff: fatal error: 'U-T' is not a valid font family
[pid 3048720] +++ exited with 1 +++
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=3048720, si_uid=1000, si_status=1, si_utime=0, si_stime=0} ---

[derij@pip build (master)]$ ll /usr/share/groff/1.23.0/font/devpdf/U-T*
-rw-r--r-- 1 root root 26563 Feb  5 23:44 /usr/share/groff/1.23.0/font/devpdf/U-TB
-rw-r--r-- 1 root root 30421 Feb  5 23:44 /usr/share/groff/1.23.0/font/devpdf/U-TBI
-rw-r--r-- 1 root root 30662 Feb  5 23:44 /usr/share/groff/1.23.0/font/devpdf/U-TI
-rw-r--r-- 1 root root 17000 Feb  5 23:44 /usr/share/groff/1.23.0/font/devpdf/U-TR


Deri James <deri>
Group Member
Mon 05 Feb 2024 08:00:28 PM UTC, comment #18: 

Why is this happening?

[derij@pip Russian]$ echo "Deri" | groff -Tpdf -fU-T -z
troff: fatal error: 'U-T' is not a valid font family
[derij@pip Russian]$ ls /usr/share/groff/1.23.0/font/devpdf/U-T*
/usr/share/groff/1.23.0/font/devpdf/U-TB   /usr/share/groff/1.23.0/font/devpdf/U-TI
/usr/share/groff/1.23.0/font/devpdf/U-TBI  /usr/share/groff/1.23.0/font/devpdf/U-TR

Whilst this is clearly wrong, I wondered if you had run this new restriction past Peter. You may not be aware of this Appendix for mom:-

http://www.schaffter.ca/mom/momdoc/appendices.html#fonts

I'm not sure whether this restriction will affect mom's use of .fam, .ft and .sty.

+verbatim+

Deri James <deri>
Group Member
Mon 10 Jul 2023 08:34:09 AM UTC, comment #17: 


commit 39ffa368dc6a1de4c11cf3f4f5b8594d3c974173
Author: G. Branden Robinson <g.branden.robinson@gmail.com>
Date:   Thu May 25 05:35:25 2023 -0500

    [troff]: Validate a font family before using it.

    * src/roff/troff/env.cpp (is_family_valid): New function checks for all
      text styles (R, I, B, BI) and returns true only if the given family
      supports them all.

      (family_change): Call `is_family_valid()` on given argument.  If
      invalid, throw diagnostic and ignore `fam` request.

    * src/roff/troff/env.h (is_family_valid): Declare; make visible.

    * src/roff/troff/input.cpp (main): Call `is_family_valid()` on `-f`
      option argument.  Its invalidity is a fatal error.

    Fixes <https://savannah.gnu.org/bugs/?64155>.  Thanks to Dave Kemper for
    the report.

    Tested with:

    $ echo | ./build/test-groff -fZD -z
    troff: fatal error: 'ZD' is not a valid font family
    $ echo | ./build/test-groff -fH -z
    $ nl ./EXPERIMENTS/validate-family.groff
         1  .tm .fam=\n[.fam]
         2  .fam H
         3  .tm .fam=\n[.fam]
         4  .fam
         5  .tm .fam=\n[.fam]
         6  .fam ZD
         7  .fam BOGUS
         8  .tm .fam=\n[.fam]
         9  .fam
        10  .tm .fam=\n[.fam]
    $ ./build/test-groff ./EXPERIMENTS/validate-family.groff
    .fam=T
    .fam=H
    .fam=T
    troff:./EXPERIMENTS/validate-family.groff:6: error: 'ZD' is not a valid font family
    troff:./EXPERIMENTS/validate-family.groff:7: error: 'BOGUS' is not a valid font family
    .fam=T
    .fam=H


G. Branden Robinson <gbranden>
Group administrator
Thu 01 Jun 2023 02:50:39 PM UTC, comment #16: 

Fixed on my branch.

And oh, look, a bug number typo in the commit message.  Ah, but I can fix it when merging to master.


commit 1fea0c142fae929cce003d6e529e000687149735
Author: G. Branden Robinson <g.branden.robinson@gmail.com>
Date:   Thu May 25 05:35:25 2023 -0500

    [troff]: Validate a font family before using it.

    * src/roff/troff/env.cpp (is_family_valid): New function checks for all
      text styles (R, I, B, BI) and returns true only if the given family
      supports them all.

      (family_change): Call `is_family_valid()` on given argument.  If
      invalid, throw diagnostic and ignore `fam` request.

    * src/roff/troff/env.h (is_family_valid): Declare; make visible.

    * src/roff/troff/input.cpp (main): Call `is_family_valid()` on `-f`
      option argument.  Its invalidity is a fatal error.

    Fixes <https://savannah.gnu.org/bugs/?64115>.  Thanks to Dave
    Kemper for the report.

    Tested with:

    $ echo | ./build/test-groff -fZD -z
    troff: fatal error: 'ZD' is not a valid font family
    $ echo | ./build/test-groff -fH -z
    $ nl ./EXPERIMENTS/validate-family.groff
         1  .tm .fam=\n[.fam]
         2  .fam H
         3  .tm .fam=\n[.fam]
         4  .fam
         5  .tm .fam=\n[.fam]
         6  .fam ZD
         7  .fam BOGUS
         8  .tm .fam=\n[.fam]
         9  .fam
        10  .tm .fam=\n[.fam]
    $ ./build/test-groff ./EXPERIMENTS/validate-family.groff
    .fam=T
    .fam=H
    .fam=T
    troff:./EXPERIMENTS/validate-family.groff:6: error: 'ZD' is not a valid font family
    troff:./EXPERIMENTS/validate-family.groff:7: error: 'BOGUS' is not a valid font family
    .fam=T
    .fam=H


G. Branden Robinson <gbranden>
Group administrator
Thu 25 May 2023 03:31:05 PM UTC, comment #15: 

In my working copy of my private branch.  The commit ID will almost certainly change.


commit f8e966af2985f267ed1efd9c6eeb37e41adca959
Author: G. Branden Robinson <g.branden.robinson@gmail.com>
Date:   Thu May 25 05:35:25 2023 -0500

    [troff]: Validate a font family before using it.

    * src/roff/troff/env.cpp (is_family_valid): New function checks for all
      text styles (R, I, B, BI) and returns true only if the given family
      supports them all.

      (family_change): Call `is_family_valid()` on given argument.  If
      invalid, throw diagnostic and ignore `fam` request.

    * src/roff/troff/env.h (is_family_valid): Declare; make visible.

    * src/roff/troff/input.cpp (main): Call `is_family_valid()` on `-f`
      option argument.  Its invalidity is a fatal error.

    Fixes <https://savannah.gnu.org/bugs/?64115>.  Thanks to Dave
    Kemper for the report.

    Tested with:

    $ echo | ./build/test-groff -fZD -z
    troff: fatal error: 'ZD' is not a valid font family
    $ echo | ./build/test-groff -fH -z
    $ nl ./EXPERIMENTS/validate-family.groff
         1  .tm .fam=\n[.fam]
         2  .fam H
         3  .tm .fam=\n[.fam]
         4  .fam
         5  .tm .fam=\n[.fam]
         6  .fam ZD
         7  .fam BOGUS
         8  .tm .fam=\n[.fam]
         9  .fam
        10  .tm .fam=\n[.fam]
    $ ./build/test-groff ./EXPERIMENTS/validate-family.groff
    .fam=T
    .fam=H
    .fam=T
    troff:./EXPERIMENTS/validate-family.groff:6: error: 'ZD' is not a valid font family
    troff:./EXPERIMENTS/validate-family.groff:7: error: 'BOGUS' is not a valid font family
    .fam=T
    .fam=H


I did have to lose the bit of diagnostic about which style was missing, because the function needed to be callable from two different places and passing back that information promised to be a pain in the butt.  It would have meant adding a reference or pointer argument to `is_family_valid`, or introducing C++ exceptions to groff's exception-free code base.  Or something similarly grody, I expect.  It didn't seem worth the trouble at this time.

G. Branden Robinson <gbranden>
Group administrator
Thu 25 May 2023 11:47:31 AM UTC, comment #14: 

Sounds good, then.  The output of comment #10 looks like it addresses the problem.  Thanks!

Dave <barx>
Group Member
Thu 25 May 2023 11:34:02 AM UTC, comment #13: 


comment #12:

> comment #11:
> > +       * src/roff/troff/env.cpp (is_family_valid): New function checks
> > +       for all text styles (R, I, B, BI) and returns true only if the
> > +       given family supports them all.
> > +       (family_change): Call `is_family_valid()` on given argument.
> > +       If invalid, throw diagnostic and ignore `fam` request.
>
> If comment #9 is correct, wouldn't this disallow a CJK family?  Or do those fonts not use the family concept?


They don't, yet.  If they're going to, we need to make the "styles" concept considerably more flexible, and probably have it driven by new directives in the device and/or font description files.  I've had a couple of different ideas along these lines, and I don't like either one.

So I reckon we can cross that bridge later.

G. Branden Robinson <gbranden>
Group administrator
Thu 25 May 2023 11:30:01 AM UTC, comment #12: 

comment #11:

> +       * src/roff/troff/env.cpp (is_family_valid): New function checks
> +       for all text styles (R, I, B, BI) and returns true only if the
> +       given family supports them all.
> +       (family_change): Call `is_family_valid()` on given argument.
> +       If invalid, throw diagnostic and ignore `fam` request.


If comment #9 is correct, wouldn't this disallow a CJK family?  Or do those fonts not use the family concept?

Dave <barx>
Group Member
Thu 25 May 2023 10:48:23 AM UTC, comment #11: 

Here's what I've pushed to my working copy.


diff --git a/ChangeLog b/ChangeLog
index 79163d005..b91609090 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,19 @@
+2023-05-25  G. Branden Robinson <g.branden.robinson@gmail.com>
+
+       [troff]: Validate a font family before trying to use it.
+
+       * src/roff/troff/env.cpp (is_family_valid): New function checks
+       for all text styles (R, I, B, BI) and returns true only if the
+       given family supports them all.
+       (family_change): Call `is_family_valid()` on given argument.
+       If invalid, throw diagnostic and ignore `fam` request.
+       * src/roff/troff/env.h (is_family_valid): Declare; make visible.
+       * src/roff/troff/input.cpp (main): Call `is_family_valid()` on
+       `-f` option argument.  Its invalidity is a fatal error.
+
+       Fixes <https://savannah.gnu.org/bugs/?64115>.  Thanks to Dave
+       Kemper for the report.
+
 2023-05-25  G. Branden Robinson <g.branden.robinson@gmail.com>

        * src/roff/troff/env.h (read_hyphen_file): Drop relic prototype
diff --git a/src/roff/troff/env.cpp b/src/roff/troff/env.cpp
index 4c64273ef..fc14cb715 100644
--- a/src/roff/troff/env.cpp
+++ b/src/roff/troff/env.cpp
@@ -1256,9 +1256,32 @@ void font_change()
   skip_line();
 }

+bool is_family_valid(const char *fam)
+{
+  // std::vector<const char *> styles{"R", "I", "B", "BI"}; // C++11
+  std::vector<const char *> styles;
+  styles.reserve(4);
+  styles.push_back("R");
+  styles.push_back("I");
+  styles.push_back("B");
+  styles.push_back("BI");
+  std::vector<const char *>::iterator style;
+  // for (auto style : styles) // C++11
+  for (style = styles.begin(); style != styles.end(); style++)
+    if (!check_font(fam, *style))
+      return false;
+  return true;
+}
+
 void family_change()
 {
   symbol s = get_name();
+  if (s != 0 /* nullptr */)
+    if (!is_family_valid(s.contents())) {
+      error("'%1' is not a valid font family", s.contents());
+      skip_line();
+      return;
+    }
   curenv->set_family(s);
   skip_line();
 }
diff --git a/src/roff/troff/env.h b/src/roff/troff/env.h
index d98941914..8382d11ea 100644
--- a/src/roff/troff/env.h
+++ b/src/roff/troff/env.h
@@ -408,6 +408,7 @@ extern void pop_env();
 extern void push_env(int);

 void init_environments();
+bool is_family_valid(const char *);

 extern double spread_limit;

diff --git a/src/roff/troff/input.cpp b/src/roff/troff/input.cpp
index cbb8b1ca3..56c20f3eb 100644
--- a/src/roff/troff/input.cpp
+++ b/src/roff/troff/input.cpp
@@ -8333,6 +8333,9 @@ int main(int argc, char **argv)
        warning(WARN_FONT, "cannot mount font '%1' directed by 'DESC'"
                " file for device '%2'", font::font_name_table[i],
                device);
+  if (fflag && !(is_family_valid(default_family.contents())))
+      fatal("'%1' is not a valid font family",
+           default_family.contents());
   curdiv = topdiv = new top_level_diversion;
   if (nflag)
     topdiv->set_next_page_number(next_page_number);


G. Branden Robinson <gbranden>
Group administrator
Thu 25 May 2023 09:46:01 AM UTC, comment #10: 

Hi Dave,

How does this look?  (Apart from a little confusing, since this is mixed input and output.)


$ ./build/test-groff -b -ww
.tm .fam=\n[.fam]
.fam=T
.fam H
.tm .fam=\n[.fam]
.fam=H
.fam
.tm .fam=\n[.fam]
.fam=T
.fam ZD
troff: backtrace: file '<standard input>':6
troff:<standard input>:6: error: 'ZD' is not a family; missing style 'I'
.fam BOGUS
troff: backtrace: file '<standard input>':7
troff:<standard input>:7: error: 'BOGUS' is not a family; missing style 'R'
.tm .fam=\n[.fam]
.fam=T
.fam
.tm .fam=\n[.fam]
.fam=H


G. Branden Robinson <gbranden>
Group administrator
Sun 07 May 2023 05:05:31 PM UTC, comment #9: 

I don't think CJK fonts have I or BI variants (but they can have R and B). To view Japanese man pages something like this is used:-

groff -Tpdf -man -dHF=MB -kt -mja -fM -dpaper=a4 -P-pa4 groff.7  > groff.7.pdf

So it probably would be better to check if the R font of given -f fam contains latin-1 charset (don't need to check all - if 'a' and 'A' are in their latin-1 positions it would be sufficiently indicative).

NB gropdf support for multi thousand glyph fonts is in development, grops already supports.

Deri James <deri>
Group Member
Sun 07 May 2023 04:25:31 PM UTC, comment #8: 


comment #7:

>   Thus "ZD" is accepted as a family name,
> as the letter "R" is interpreted as meaning a roman font.


[and on the 'ps' device, a "ZDR" font description actually exists]

I noticed this myself, and is one reason I proposed checking all four styles when the user (attempts to) select a family.

G. Branden Robinson <gbranden>
Group administrator
Sun 07 May 2023 02:56:24 PM UTC, comment #7: 

  My "solution" has a side effect,
so it can't be integrated into the software.

  If an input file is used with some characters,
then it turns out,
that the font "ZDR" is used.

  Thus "ZD" is accepted as a family name,
as the letter "R" is interpreted as meaning a roman font.

Bjarni Ingi Gislason <bjarniig>
Sun 07 May 2023 02:35:41 AM UTC, comment #6: 


> Is it an error to specify a family where some font styles are missing if the document never asks to use those styles?


I'd characterize that as a warning.  I'd like to see the formatter check for the existence of all four styles when a family is requested.  (Or at startup if `-f` is specified.)
 

> Is it an error to specify a family where all font styles are missing if the document never actually uses that family? (e.g., specifying "-fZD" on the command line to process a document whose first line is ".fam T")


I'd make this a warning too.

And, more realistically, in the case where, on the "dvi" device, someone specified -fC, thinking they could get something equivalent to the "Courier" family.  But the Computer Modern fonts aren't quite up to that.

G. Branden Robinson <gbranden>
Group administrator
Sun 07 May 2023 01:57:25 AM UTC, comment #5: 

comment #3:

> Possibly a search for the requisite styles could be undertaken
> upon use of the `-f` flag or `fam` request, errors thrown for
> any that are missing, and the option or request ignored.


Hmm, that raises a couple of interesting questions.

Is it an error to specify a family where some font styles are missing if the document never asks to use those styles?

Is it an error to specify a family where all font styles are missing if the document never actually uses that family? (e.g., specifying "-fZD" on the command line to process a document whose first line is ".fam T")

Dave <barx>
Group Member
Sun 07 May 2023 01:38:45 AM UTC, comment #4: 

comment #3:

> However, Zapf Dingbats is a specific face, not a family.  And
> not a member of a family.


Whoops!  You're right.

> Remember this old chestnut?
>


>     troff: EXPERIMENTS/font-errors.groff:23: warning: can't find font 'BOGUSR'


>
> This is pretty close to the actual problem.  If "ZD" is the
> family, then the formatter expects to find fonts named "ZDR",
> "ZDI", "ZDB", and "ZDBI".


This could be what's going on, but the "character not defined" warnings suggest that groff is successfully (though perhaps wrongly) setting the font to something that lacks basic alphabetic characters (as ZD does), then running into the string-comparison issue Deri noted (comment #1).  If groff were seeking a phantom "ZDR" font, the font-setting should fail altogether, leaving the font as TR, which I wouldn't expect to produce those particular warnings.

> The numeric overflow is interesting.


#NoBoringBugs

Dave <barx>
Group Member
Sat 06 May 2023 05:16:42 PM UTC, comment #3: 

This is kind of a case of "don't do that, then".  ("Doctor, it hurts when I do this!")


    -f fam
        Set default font family.


However, Zapf Dingbats is a specific face, not a family.  And not a member of a family.

No matter what happens, you'll get diagnostics, but we probably could improve the diagnostics you get.

Remember this old chestnut?


    troff: EXPERIMENTS/font-errors.groff:23: warning: can't find font 'BOGUSR'


This is pretty close to the actual problem.  If "ZD" is the family, then the formatter expects to find fonts named "ZDR", "ZDI", "ZDB", and "ZDBI".

Possibly a search for the requisite styles could be undertaken upon use of the `-f` flag or `fam` request, errors thrown for any that are missing, and the option or request ignored.

The numeric overflow is interesting.

G. Branden Robinson <gbranden>
Group administrator
Sat 06 May 2023 04:57:10 PM UTC, comment #2: 

  I found a solution, it did though need thinking to find it.

test-groff -ww -b -z -fZD /dev/null

  runs then with no output and returns with zero as exit value.

  I will leave this bug as is, to give you (plural) a chance to find a
solution on your own.

Bjarni Ingi Gislason <bjarniig>
Sat 06 May 2023 12:20:15 PM UTC, comment #1: 

I suspect this is related to bug #64144 (errors when setting -f to a font which does not contain a full latin-1 character set). It occurs when using .if ''' in a macro so a construct such as .if '\*[.T]'html' will fail when troff generates the output of both to compare.

Branden devised an ingenious solution in bug #64144, it is a bit odd because -T pdf produces a different error:-

[derij@pip build (new-gropdf)]$ echo ' ' | ./test-groff -Tps -fEURO -a
troff: fatal error: invalid default font family 'EURO'
[derij@pip build (new-gropdf)]$ echo ' ' | ./test-groff -Tpdf -fEURO -a
troff: fatal error: invalid default font family 'EURO'
[derij@pip build (new-gropdf)]$ echo ' ' | ./test-groff -Tpdf -fZD -a
troff: fatal error: invalid default font family 'ZD'
[derij@pip build (new-gropdf)]$ echo ' ' | ./test-groff -Tps -fZD -a
troff:/home/derij/groff-git/groff/build/../tmac/fallbacks.tmac:14: warning: character 'p' not defined
troff:/home/derij/groff-git/groff/build/../tmac/fallbacks.tmac:14: warning: character 's' not defined
troff:/home/derij/groff-git/groff/build/../tmac/fallbacks.tmac:14: warning: character 'a' not defined
troff:/home/derij/groff-git/groff/build/../tmac/fallbacks.tmac:14: warning: character 'c' not defined
troff:/home/derij/groff-git/groff/build/../tmac/fallbacks.tmac:14: warning: character 'i' not defined
troff:/home/derij/groff-git/groff/build/../tmac/fallbacks.tmac:16: warning: character 'l' not defined
troff:/home/derij/groff-git/groff/build/../tmac/fallbacks.tmac:16: warning: character 't' not defined
troff:/home/derij/groff-git/groff/build/../tmac/fallbacks.tmac:16: warning: character 'n' not defined
troff:/home/derij/groff-git/groff/build/../tmac/fallbacks.tmac:17: warning: character 'u' not defined
troff:/home/derij/groff-git/groff/build/../tmac/fallbacks.tmac:17: warning: character 'f' not defined
troff:/home/derij/groff-git/groff/build/../tmac/troffrc-end:7: warning: character 'h' not defined
troff:/home/derij/groff-git/groff/build/../tmac/troffrc-end:7: warning: character 'm' not defined
<beginning of page>

troff:/home/derij/groff-git/groff/build/../tmac/html-end.tmac:20: error: numeric overflow


Deri James <deri>
Group Member
Fri 05 May 2023 11:41:19 PM UTC, original submission:  

This bug was present in 1.22.4:


$ echo ' ' | groff -fZD -a
troff: /usr/share/groff/1.22.4/tmac/ps.tmac:686: warning: can't find character 'a'
troff: /usr/share/groff/1.22.4/tmac/troffrc-end:7: warning: can't find character 'p'
troff: /usr/share/groff/1.22.4/tmac/troffrc-end:7: warning: can't find character 's'
troff: /usr/share/groff/1.22.4/tmac/troffrc-end:7: warning: can't find character 'h'
troff: /usr/share/groff/1.22.4/tmac/troffrc-end:7: warning: can't find character 't'
troff: /usr/share/groff/1.22.4/tmac/troffrc-end:7: warning: can't find character 'm'
troff: /usr/share/groff/1.22.4/tmac/troffrc-end:7: warning: can't find character 'l'
<beginning of page>

troff: /usr/share/groff/1.22.4/tmac/html-end.tmac:20: numeric overflow


But the number of warnings increased in 1.23.  (Pathnames below trimmed.)

$ groff --version | head -1
GNU groff version 1.23.0.rc4.19-96b92-dirty
$ echo ' ' | groff -fZD -a
troff:.../tmac/fallbacks.tmac:14: warning: character 'p' not defined
troff:.../tmac/fallbacks.tmac:14: warning: character 's' not defined
troff:.../tmac/fallbacks.tmac:14: warning: character 'a' not defined
troff:.../tmac/fallbacks.tmac:14: warning: character 'c' not defined
troff:.../tmac/fallbacks.tmac:14: warning: character 'i' not defined
troff:.../tmac/fallbacks.tmac:16: warning: character 'l' not defined
troff:.../tmac/fallbacks.tmac:16: warning: character 't' not defined
troff:.../tmac/fallbacks.tmac:16: warning: character 'n' not defined
troff:.../tmac/fallbacks.tmac:17: warning: character 'u' not defined
troff:.../tmac/fallbacks.tmac:17: warning: character 'f' not defined
troff:.../tmac/troffrc-end:7: warning: character 'h' not defined
troff:.../tmac/troffrc-end:7: warning: character 'm' not defined
<beginning of page>

troff:.../tmac/html-end.tmac:20: error: numeric overflow


Dave <barx>
Group Member

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by PTPi (Posted a comment)
  • -email is unavailable- added by gbranden (mom author)
  • -email is unavailable- added by gbranden (Posted a comment)
  • -email is unavailable- added by bjarniig (Posted a comment)
  • -email is unavailable- added by deri (Posted a comment)
  • -email is unavailable- added by barx (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 25 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2024-05-20 barx CategoryCore Macro package - others/general
        Summary[troff] specifying -fZD on command line generates warnings Specifying -fZD on command line generates warnings
    2024-05-14 gbranden StatusIn Progress Fixed
        Open/ClosedOpen Closed
    2024-05-11 gbranden StatusConfirmed In Progress
    2024-05-03 gbranden CategoryFont - others/general Core
        StatusNeed Info Confirmed
        Summaryspecifying -fZD on command line generates warnings [troff] specifying -fZD on command line generates warnings
    2024-05-03 PTPi StatusConfirmed Need Info
    2024-05-03 gbranden StatusNeed Info Confirmed
    2024-05-02 gbranden StatusConfirmed Need Info
    2024-03-06 barx StatusNeed Info Confirmed
        Assigned toPTPi gbranden
    2024-03-05 barx Assigned toderi PTPi
    2024-02-06 gbranden Assigned togbranden deri
        Carbon-Copy- Added -email is unavailable-
    2024-02-06 deri Assigned toderi gbranden
    2024-02-06 gbranden Assigned togbranden deri
    2024-02-05 deri Severity2 - Minor 3 - Normal
        StatusFixed Need Info
        Open/ClosedClosed Open
    2023-07-10 gbranden StatusReady for Merge Fixed
        Open/ClosedOpen Closed
        Planned ReleaseNone 1.24.0
        Summary[PATCH] specifying -fZD on command line generates warnings specifying -fZD on command line generates warnings

    Back to the top

    Powered by Savane 3.13-b921.
    Corresponding source code