comment #15:

> A little detangling from mom(7) will be necessary.

I retract that; mom can still use Keith's pdfmark no matter where it comes from, and removing contrib/pdfmark causes no mom test failures, so I see no reason to alter mom at this time.

A little detangling from mom(7) will be necessary.

$ git grep 'gropdf-info'
contrib/mom/om.tmac:.              if !dpdf:href.map .tm gropdf-info:href \\*[PDF_NM2] \\*[PDF_TX]
contrib/mom/om.tmac:.    if '\\*[.T]'ps' .if !dpdf:href.map .tm gropdf-info:href \\*[ARG_1] \\$*
contrib/pdfmark/pdfmark.tmac:.ie '\\n(.z'' .tm gropdf-info:href \\*[PDFHREF.NAME] \\$*
contrib/pdfmark/pdfroff.sh:    $AWK '/^gropdf-info:href/ {$1 = ".pdfhref D -N"; print}' $WRKFILE > $REFFILE

I'm wondering how much we really have to do differently for the "ps" and "pdf" devices.  Since "pdfmark" commands have their device controls prefixed with "ps: exec", I wonder if they're syntactically valid PostScript language that just happens not to be interpreted by most PostScript interpreters.  If that's correct, it would explain why transformations à la the ps2pdf(1) command are not more lossy than they are.

Deri, does that sound right to you?

comment #11:

> comment #10:
> > Should the 1.23 announcement/news mention that:
> > * the pdfmark bundled with this release of groff is not the latest code (and provide a pointer to that code)?
>
> I don't think they've diverged all that much.

Keith addressed this in comment #12; I don't know if that changes the conclusion any.

> pdfroff(1) will continue to exist in groff 1.23.0
> distributions and point readers to Keith's site.

Ah, yes, I somehow missed commit 948ccc55, even though it was cited in a comment to bug #63133, on which I'm cc:ed.

comment #10:

> Should the 1.23 announcement/news mention that:
> * the pdfmark bundled with this release of groff is not the latest code (and provide a pointer to that code)?

I don't think they've diverged all that much.  (Time will of course tell how much they have done in a year or so.)  pdfroff(1) will continue to exist in groff 1.23.0 distributions and point readers to Keith's site.

> * this is the last groff release that will have pdfmark bundled with it?

No, I don't care to give a hostage to fortune in that matter.  As it happens I'm looking at withdrawing contrib/pdfmark on a branch right now, but I might not finish that work or it might end up not happening for whatever reason.

I'd prefer the news and announcement files to state things that are known as of the time of release, rather than risking baking inaccuracies in them.  (Beyond those errors that inevitably arise after 20 people have proofread something 20 times but only rear their heads after going to press.)

comment #8:

> comment #6:
> > comment #3:
> > > ...
> > > groff's man pages all have that logic at the top and bottom that
> > > takes the body of the page out of compatibility mode, ...
> > As already noted in bug #63133 comment #1, your code, to do that:
> [snip]
> > doesn't work on groff-1.22.x and earlier, (because those older
> > groff versions do not grok the 'cp' register).
>
> [suggested replacement]:

> .\" Save and disable compatibility mode (e.g., for Solaris 10/11).
> .nr _C \n(.C
> .do rnn _C *groff_pdfroff_1_man_C
> .cp 0

>
> This is similar to something I've had in my man.local for quite a while.
>

> .if \n(.x=1&\n(.y<23 \{
> .  \" Construct an equivalent of groff 1.23's .cp register.
> .  nr }{ \n(.C
> .  do nr .cp \n(}{
> .\}

>
> I'm not sure which I like better.  Yours does have appeal.

Either should work, I think.  As you might expect, I tend to favour mine, not just because it is mine, but because:

• It doesn't introduce any conditional version dependency; (feature test conditions should always trump version tests, but it doesn't even depend of a feature test).
• It doesn't pollute the register namespace with a 'cp' register, which wouldn't otherwise be present, and shouldn't be needed, (unless there is another version dependency, within the document stream, which requires it).

comment #5:

> comment #3:
> > comment #2:
> > > ...
> > > More importantly, my simple implementation relies on the '!d'
> > > conditional, which I suspect may be groff-specific, (is it?),
> >
> > Yes.  Subsection "Conditional expressions" of groff_diff(7) covers
> > it, and the other extensions.
> Thanks, but there is no such subsection, in the version of groff_diff(7) on my Manjaro (Arch Linux) system.

I'm sorry, I didn't mention that I was referring to the Git version, which is what I use every day.

The subsection has been present since commit 097b3c6c4e in May 2020.

> The information is there, but not so easily found.  In any event, I had been perusing 'info groff', which -- again in the installed version -- does not make it clear that '.if d...' is a groff extension, at the point where the conditional operators are described.

You're right, groff's Texinfo manual generally doesn't dwell on what is or is not a GNU extension.  That was the style it had when Trent and Werner stopped working on it, and I've preserved that aspect except for deeper syntactical extensions.

> Provided the implementation which does exist will DTRT, w.r.t. the expectations from using the groff >= 1.23 implementation, sure.  But if you don't know for sure that any existing implementation mimics groff >= 1.23 behaviour, all bets are off; you surely do want to clobber it.

I am not in fact so sure about this.  Your argument could be extended to every man(7) macro groff supports.  I would regard the inlining of the entire macro package (excepting maybe `TH` alone) in the prologue of every man page as too costly.

I want groff's man pages to serve as examples for other man page writers to follow, and that would be contrary to the goal, bewildering the novice writer with an avalanche of details.
 

> > groff's man pages all have that logic at the top and bottom that
> > takes the body of the page out of compatibility mode,
> Sure, but what if that compatibility save, disable, then restore hack doesn't work?

This isn't a hypothetical; when multiple man pages were rendered, prior to early in the groff 1.23.0 development cycle, it in fact never did work; see bug #58162.

>  If the page source is passed through a formatter which doesn't support the 'do' and 'cp' requests, or otherwise support groff extended syntax, then all kinds of output garbage may ensue.

That's going to be true even if we don't bother trying to turn on compatibility mode at all (or only do it inside our macro definitions), because non-CSTR #54 syntax is all over the pages anyway, as I noted in comment #3 and comment #4.

> > and the pages generally use groff extensions like special characters
> > that aren't documented in CSTR #54, often using the \[foo] syntax
> > that also wasn't supported by AT&T troff.
> In which case, attempting to retain even a modicum of support for legacy formatting engines is likely a forlorn venture, so why bother?

A couple or three years ago, Ingo was putting a lot of pressure on me to not "break portability".  At the time I didn't have as clear an idea of what formatters were out there actually being used, and how groff's man pages had been falling short of his personal standard for portability for decades--in many cases since James Clark first wrote them.

Solaris 10 troff will be end-of-lifed along with the rest of it in about 1 year--assuming they stick to their announced plan, so call it a 50/50 chance.  Solaris 11 uses groff.  I don't think anyone actually uses DWB 3.3 troff as a daily driver.  Plan 9 from User Space troff is still developed, but is fairly recalcitrant to innovations, apart from adding the `MR` man(7) macro.  This may be due to lack of attention or a low developer head count.

So in early 2024 it may be time to look again at updating the backward time horizon for formatters groff expects its man pages to be processed by.

> I'm inclined to disagree, only to the extent that any legacy formatter may be confused by them, but maybe I'm just being excessively pedantic.

I take the legacy formatter support question seriously insofar as I have any actual evidence that legacy formatters are being used.  If we don't concern ourselves with the empirically measurable, then your same argument can be employed to argue against the use of features introduced by Kernighan in ~1981 to device-independent troff, or really to anything subsequent to the first nroff in 1972 or so.

As a case in point, and as I have recently documented (i.e., don't bother looking for it in your Manjaro man pages), the `SB` macro was an extension.  It was not in Unix Version 7 man(7) in 1979, the first implementation.  It wasn't in any man(7) until about 1988, when SunOS 4 introduced it.  James Clark swiftly cloned it in groff.  As far as I know it was never implemented in any other proprietary troff.  But the maintainers of the Korn Shell, which ran on just about every Unix, employed it in their ksh(1) man page early on, never looked back, and if any users complained about its unportability, they were apparently ignored.

This should have offended Ingo Schwarze and prompted one of his prophecies of doom; it's just as bad as my/P9US's `MR` because text will be silently omitted if the macro isn't supported.  (He has a slightly higher tolerance level for macros that don't take text arguments.)

But what has happened in practice?  Apparently, no one noticed or cared, and a lot of people assumed it was "standard" when it wasn't.  Plenty of people moved away from ksh to something else, but I've never heard a complaint that the unportable man page was the reason.  (There were much better ones, from licensing to feature creep.  Too much of it got standardized in POSIX as it was, and the hasty adoption of aliases as a standard feature has caused problems literally to this very day.  Yes, I mean 24 February 2023.)

> > The idea is to get something that works on Heirloom Doctools troff,
> > mandoc, and older versions of groff.  This doesn't work perfectly in
> > general, but for the MR feature specifically, it does.

> Fair enough, provided we're happy to abandon any hope of legacy formatter support otherwise.

I think whether we do, or have done this, depends on how one defines "legacy formatter support".

Breaking the renderability of man pages on a system that someone does real work on is an anti-goal.  Of mine, and I assume of others.

Breaking the renderability of man pages on systems that are only run for historical research, I may find a regrettable necessity; I won't break them for fun, but if I implement any change to groff (or man(7)) I usually have a reason that is compelling to me because doing so leads to lengthy discussions like this one where I defend my decisions.

Breaking the renderability of man pages to systems that are defunct, often unrecoverably so due to lost hardware or software, or to theoretical or future Puritan re-implementations of historical systems, I am indifferent to.  I encourage people to write their own text formatters as a couple of Kernighan's books suggested.  But a system for deployment to the real world has to deal with real world specimens of input.  groff's man pages are as valid in that sense as anyone else's.  More than most, I would venture--if you read down mandoc(1)'s NEWS file, you will find numerous features, many well down into the weeds of the formatter, that Ingo or Kristaps before him had to implement to sensibly process man pages not written by groff's developers.

Put much more simply, I want to avoid regressing portability, which implies a measurable baseline for comparison--real code being used by real people.  Objections to new "unportabilities" because they negatively affect unwritten formatters or nonexistent users should not be credited.  There may be other reasons to object to a feature, such as poor design or implementation, but deploying unsupported claims of unportability is fallacious argument.

I trust you will note that when Alexis demonstrated on the groff list that I had broken portability to real systems, rather than arguing with him as I am doing with you, I responded within hours, having reproduced the problem and determined that it was a show-stopper (release-blocker).  Within 24 hours I had the change reverted and replaced with one that works.

That's the power of empirical measurement and reproducible results.

comment #3:

> comment #2:
> > ...
> > More importantly, my simple implementation relies on the '!d'
> > conditional, which I suspect may be groff-specific, (is it?),
>
> Yes.  Subsection "Conditional expressions" of groff_diff(7) covers
> it, and the other extensions.

Thanks, but there is no such subsection, in the version of groff_diff(7) on my Manjaro (Arch Linux) system.  The information is there, but not so easily found.  In any event, I had been perusing 'info groff', which -- again in the installed version -- does not make it clear that '.if d...' is a groff extension, at the point where the conditional operators are described.

> ...
> > if we are concerned about non-groff compatibility,
>
> The test is not so much for implementation of groff extensions as
> the likelihood of the `MR` macro being already defined by that
> implementation.  If one exists, I don't want to clobber it because
> it can do much more than just typeset the arguments.  It can embed
> hyperlinks on supporting output devices.

Provided the implementation which does exist will DTRT, w.r.t. the expectations from using the groff >= 1.23 implementation, sure.  But if you don't know for sure that any existing implementation mimics groff >= 1.23 behaviour, all bets are off; you surely do want to clobber it.

> groff's man pages all have that logic at the top and bottom that
> takes the body of the page out of compatibility mode,

Sure, but what if that compatibility save, disable, then restore hack doesn't work?  If the page source is passed through a formatter which doesn't support the 'do' and 'cp' requests, or otherwise support groff extended syntax, then all kinds of output garbage may ensue.

> and the pages generally use groff extensions like special characters
> that aren't documented in CSTR #54, often using the \[foo] syntax
> that also wasn't supported by AT&T troff.

In which case, attempting to retain even a modicum of support for legacy formatting engines is likely a forlorn venture, so why bother?

> > can we safely use long register names, such as your 'do-fallback'?
>
> I think we can, because Heirloom, mandoc(1), and older groffs
> support them.

I'm inclined to disagree, only to the extent that any legacy formatter may be confused by them, but maybe I'm just being excessively pedantic.

> The idea is to get something that works on Heirloom Doctools troff,
> mandoc, and older versions of groff.  This doesn't work perfectly in
> general, but for the MR feature specifically, it does.

Fair enough, provided we're happy to abandon any hope of legacy formatter support otherwise.

comment #2:

> Granted, that doesn't address the possibility that MR might be invoked with fewer than two arguments -- because pdfroff.1.man never calls it with fewer -- but that's a trivial adjustment, for the more general case.

Right.

> More importantly, my simple implementation relies on the '!d' conditional, which I suspect may be groff-specific, (is it?),

Yes.  Subsection "Conditional expressions" of groff_diff(7) covers it, and the other extensions.

> in which case, I need to address that.  However, on a related note, if we are concerned about non-groff compatibility,

The test is not so much for implementation of groff extensions as the likelihood of the `MR` macro being already defined by that implementation.  If one exists, I don't want to clobber it because it can do much more than just typeset the arguments.  It can embed hyperlinks on supporting output devices.

groff's man pages all have that logic at the top and bottom that takes the body of the page out of compatibility mode, and the pages generally use groff extensions like special characters that aren't documented in CSTR #54, often using the \[foo] syntax that also wasn't supported by AT&T troff.

groff_man*(7) is a little more conservative--though I have repeatedly questioned the worth of treating it any differently.

> can we safely use long register names, such as your 'do-fallback'?

I think we can, because Heirloom, mandoc(1), and older groffs support them.

The idea is to get something that works on Heirloom Doctools troff, mandoc, and older versions of groff.  This doesn't work perfectly in general, but for the MR feature specifically, it does.

For what it's worth, the pdfroff(1) man page in groff 1.22.3 (before I showed up) uses the following "groffisms".

• groff-style special character escape sequences \[co]
• groff-style string interpolation escape sequences \*[lq], \*[rq]
• non-hyphenated break point escape sequence \:
• groff man(7) extension macros: UR, UE, MT, ME, SY, YS
• "all arguments" interpolation escape sequence \*
• special character identifiers: oq, cq, aq

It also uses the following features which Ingo Schwarze recommends against and may not work, or not work fully, in mandoc(1).

• macro definition: de
• hyphenation exception word request: hw
• alteration of hyphenation mode: hy

This old pdfroff(1) also sets the hyphenation mode to 1 at the end of the `NH` definition (implicitly).  Unfortunately, there is no portable way to access or store the current hyphenation mode.  Further the mode numbers are not portable across implementations; not because they have different meanings (though groff has extended values not supported by AT&T) but because the hyphenation patterns themselves have different expectations about where the possible hyphenation points in words are.  (The AT&T implementation assumed invariants that hold only for languages like English.)

It also uses a special character identifier that is not defined in any implementation I know of, "ok".  (Possibly a typo.)

I decided when I started working on groff's man pages, as I learned more about the capabilities of "classic" troff, that it wasn't worth trying to achieve perfect portability to Version 7 Unix troff.  Too much would have to be sacrificed.

Other man page authors and maintainers can of course have different goals.

bug #63133 comment #5:

> Bug #63827 is now open to track the removal of pdfmark from the
> groff git repository.  The following is more relevant to that bug,
> but I'm starting it here because it's a reply to a comment here,
> and Keith is already cc:ed here.
>
> comment #3:
> > Agreed.  I have, indeed, added significant new content to
> > pdfmark.pdf, (via pdfmark.ms), and some pdfroff enhancements,
> > which are not now included in groff's contrib/pdfmark tree.
>
> There have also been 12 commits that have touched groff's
> contrib/pdfmark tree since the 2021-11-13 creation of the OSDN
> pdfmark, according to:
>
> git log --since 2021-11-13 contrib/pdfmark
>
> I presume these haven't been reflected in OSDN.

Some have, albeit not in exactly the same manner; some of the others, I will address shortly.

> Many of them are part of larger changes to the groff code base,
> and may or may not be necessary

Those which impact, primarily, on the build infrastructure, are irrelevant; I do not use automake, (and will not entertain doing so); I do use GNU make, and make no apology for introducing GNU make specific constructs, within my makefile.

> to keep the two interoperating happily, but someone familiar with
> the code will have to make that call for each change.

I'll be happy to look into any concerns; please feel free to bring any to my attention, either by commenting on this ticket, or by opening one on my OSDN groff-pdfmark ticket page; (you may need to create a personal OSDN account, for the latter).

> (This presumes that the OSDN repository was created from the latest
> groff git contrib/pdfmark at the time;

It was.

> if it was spawned from another source, there might be more
> divergence.)

Everything, up to the date of the fork, I have captured.  Some changes, since then I already have captured, in some shape or form; some others, (at least one of which looks suspiciously wrong, to me), I may need to address.