Thanks for the history! I also don’t see it used anywhere really.

>By the way, do we still have that ancient README file in the groff git tree?
>I think that OpenBSD still has it lying around in the source tree in
>/usr/src/share/tmac/mdoc/README is a mistake, too.

Unsure about groff, but in MirBSD I still have it, probably from OpenBSD; as I said, my tmac is a merger of OpenBSD’s pre-mdocml and the “last” one from classic BSD, which I then made work with the last nroff under the Caldera licence. Maybe also not based on the one from OpenBSD. But if you have meaningful comments on
http://www.mirbsd.org/cvs.cgi/src/share/tmac/mdoc/README?rev=HEAD
(or the rest of tmac, really), be my guest (but perhaps not here unless it’s for cross-*roff compatibility).

Regarding editing manual page source code in such a way as to avoid particularly ugly line breaks in standard-width 80 column terminal windows:

comment #12:

> Possibly, mdoc(7) page authors knew this and carefully edited the ones that did, so that now no one sees them.
> But they would have be de-semanticizing their inputs by sweating formatting details.
> Perhaps Ingo will join me in finding that dubious,

In OpenBSD, people mostly avoid such hand-optimization of presentational markup and stick to semantic markup.  In the vast majority of cases, that yields good results and minimizes maintenance effort.

However, a smaller number of pages exists that, for one reason or another, are hard to get into a state both easy to read and looking pleasant, if you purely stick to presentational markup.  It typically  happens with content that is more complicated and harder to understand in the first place.  These cases are not typically as simple and straightforward as SYNOPSIS sections; we tend to stick to semantic markup in the SYNOPSIS.  The auto-breaking features of .Op, .Fl, and .Ar tend to work reliably in general.

I know for sure that in such cases, our chief documentation maintainer, Jason McIntyre, occasionally does resort to manually optimizing the source code such that output lines do not exceed 80 columns, do not break in bad places places, and so on.  Now if we would suddenly increase the global indentation by 2n, most of these hand-optimized cases would suddenly become hard to read and ugly in precisely the way Jason spent some work on avoiding.  That would be bad because, as i said, these are not just random cases, but typically cases with complicated content, where causing an additional distraction for the readers would be particularly unfortunate.

For that reason, i'm definitely not going to increase the global offset from 5n to 7n in mandoc(1).  Even if you were to do that in groff(1), mandoc would certainly not follow, and i might possibly even patch it back in the OpenBSD port of groff.

I'm not quite sure how this kind of hand-optimization is regarded in FreeBSD and NetBSD.  I talked to both Warren Block and Thomas Klausner multiple times face-to-face, but don't recall ever bringing up this particular topic.  I guess their view might be somewhat similar, but i'm not completely sure.  It seems likely to me their approach might be somewhat less systematic and more ad-hoc than in OpenBSD.  So i cannot exclude that hand-optimization might be slightly more common and semantic markup slightly weaker on average in their pages than in ours.  In general, they tend to invest less into documentation than we do.  I know even less about DragonflyBSD, except that they usually follow FreeBSD quite closely (even though often with significant time delays) unless they are specifically working on an area of their system, so i doubt they are doing much general-purpose manual page work in the first place.

> even if he hates the changed indentation (which I aim to change back, and port over to groff man(7), in case that wasn't clear).

I felt relieved when earlier comments in this ticket made this seem likely to me.  Thank you!

Alex's case of cringe at my `is_family_valid` function has been raised on the groff development list.

https://lists.gnu.org/archive/html/groff/2023-08/msg00008.html
https://lists.gnu.org/archive/html/groff/2023-08/msg00009.html

It is off-topic for this ticket; let's please discuss `lengthof` elsewhere.

Let me update my intentions from comment #5 as follows.

This ticket contemplates two new registers for man(7), and restricted
meaning for the `IN` register, first exposed/documented by Documenter's
Workbench (DWB) 1.0 in 1984.

* BP: base paragraph indentation configures how far the left margin of
  a normal paragraph is inset from the page offset (which is 0 on
  terminals) to which relative insets are added.  Default: 5n.
  Currently, `IN` is used for this purpose as well as `RS`.

* IN: the "standard indentation" would now configure only the default
  indentation increment used for relative inset.  Default: 7n/7.2n (no
  change).

* TS: the "tag spacing" configures the size the of the gap that is
  required after a paragraph tag.  This amount is added to the width of
  the diversion created by the `IP` and `TP` macros to format the tag.
  If that amount is greater than the standard indentation, a break is
  emitted and the remainder of the paragraph is formatted starting on a
  new output line, indented by the `IN` increment.  Default: 2n.
  Currently, a value of 1n is used for this purpose.  (In groff 1.23.0,
  it's stored in the internal `an-tag-separation` register, but not
  readily user-configurable.)

For consistent rendering of mdoc(7) pages, these parameters (if set)
would be honored by that macro package too, as `SN` and `IN` already
are, where comparable macro semantics permit.

comment #15:

> I regret that the implementation language makes groff so stinky for people.  Having seen the problems it was solving, I can understand why Clark selected it over the just-born ANSI C.  It's much closer to applications programming than systems programming, and C++ had much promise there.  Of course Stroustrup promoted as the best language for everything.  :-/
>
> From today's perspective, in groff there are huge amounts of data-structure walking code that could be replaced by C++98 "algorithms" (they are, but God the name assaults the nose like a bottle of brogrammer patchouli oil), or cleanly replaced by C++11 idioms.
>
> I get the feeling that Clark ended up not doing as much input validation as he might simply because it was so incredibly tedious to walk data structures.
>
> Here's a recent example of some validation I added, with annotations of future possibilities.
>

> bool is_family_valid(const char *fam)
> {
>   // std::vector<const char *> styles{"R", "I", "B", "BI"}; // C++11
>   const size_t nstyles = 4;
>   const char *st[nstyles] = { "R", "I", "B", "BI" };
>   std::vector<const char *> styles(st, (st + nstyles));
>   // for (auto style : styles) // C++11
>   std::vector<const char *>::iterator style;
>   for (style = styles.begin(); style != styles.end(); style++)
>     if (!check_font(fam, *style))
>       return false;
>   return true;
> }

>
> C++11 would cut this function about in half.  Of course, the whole thing is pure bloat from an Annotated Reference Manual C++ perspective, where'd you skip all this ridiculous validation entirely because what could go wrong?  The ARPAnet is such a friendly place...
> 

Wow!  You can cut that in half in C.


#define lengthof(a)  (sizeof(a) / sizeof((a)[0]))


bool is_family_valid(const char *fam)
{
        static const char  st[][3] = { "R", "I", "B", "BI" };

        for (size_t i = 0; i < lengthof(st); i++) {
                if (!check_font(fam, st[i]))
                        return false;
        }
        return true;
}

comment #14:

> > Possibly, mdoc(7) page authors knew this and carefully edited the
> > ones that did, so that now no one sees them.
>
> Exactly, we do that… I recently also begun rewording things to
> avoid hyphenation as well, although only with “french” spacing
> (no american double-space after a full stop) since that’s what
> I use in my BSD.

Dave Kemper and I (both Americans) have different views on how wide the space after a sentence should be, but we both found the following resource valuable.

https://web.archive.org/web/20171217060354/http://www.heracliteanriver.com/?p=324

groff makes this parameter tunable for all output devices, so it's not a source of strife, except among those who don't read documentation.

> >> We figured out that putting \& after punctuation only for stuff like
> >> “e.g.\&” (where you don’t want the american double-spacing after),
> >> and otherwise before (e.g. “\&.”
>
> That would be “.Dq \&.”, for the sake of completeness.
> I also saw “.Dq .\&”, and, for some time, people were
> not clear about which one to use, but in discussion
> with J�rg, it got clear that the most portable is to
> put the \& in front always except “e.g.\&”.

For the case of ".Dq .", I agree.  The macro itself won't (shouldn't) put a bare dot at the beginning of an (interpolated) input line, so the worry is that interpolation will result in the quoted dot being treated as the end of a sentence, just as:

He said, "the guys just left."

is.

> No, something like “.Dq \&Li”, that is, where I have a
> two-character argument to a parsed macro, so it isn’t
> interpreted as callable macro.
>
> This application is disctinct from escaping a leading
> dot or apostrophe or an end-of-line dot that’s not a
> full stop.

Agreed.  This is an additional application of the dummy character created by mdoc(7)'s unique design.
 

> > Interesting.  I did not know `In` was a late-breaking macro
>
> I first saw it in manpages from NetBSD, and OpenBSD did
> not have it, nor use it. (I think that before the switch
> to mdocml they didn’t change their tmacs much.)

Interesting.  Perhaps the feeling was that it wasn't worth maintaining any aspect of troff as the desire built to kill it.

> >> We cannot, obviously, have three-letter requests.
> > Nope.  Like I said, there's room for `Cq`, `Co`, and `Cc`.
>
> Indeed, I see only Co used grepping through all tmacs:
> tmac.doc.old has it as macro (just .tm’ing to say it’s
> not an mdoc macro) plus…

It's in groff's "doc-old.tmac", too, which has the same origin.

Huh.  I wonder what the story behind that is.

> | mdoc/README:.\" NS Co register (site) Width Needed for Column offset
>
> … I’m not sure if this is still true, given my grep
> did not find any other occurrence? I think this is
> old/wrong and needs to be removed.

It seems likely to me.

I would guess that Ingo has the world's biggest corpus of mdoc(7) documents readily at hand--but perhaps not the time to grep them for our benefit.  :P

> >> The codebase is the “last” nroff I could use under the Caldera
> >> licence, i.e. that was shipped with a BSD covered by these. The
>
> > Is there anyplace these can currently be obtained?
>
> I got them from minnie.tuhs.org; if your CVS skills are still
> not too rusty, you can get the subset I imported from MirBSD
> anoncvs, too.

My CVS skills are pretty rusty but this wouldn't help me.  Every time someone has pointed me to something they said was "1980s troff sources", it was a descendant of Ossanna troff, not Kernighan's rewrite.  In other words, it was 1970s troff.

I think the first time someone sent me on that goose chase it was to Kirk McKusick's CSRG CD-ROMs.  Much great stuff, but not a vintage 1981 Kernighan troff.  I want that so bad.
 

> And yes, it’s not the later one, it’s the old one where troff
> was for that one typesetter(?) machine. I do also have a
> tape archive of a ditroff predating 1990 which would be in the
> PD in the USA but not in the rest of the world, so I cannot
> use it (and trying to figure out who even could give a
> licence is probably not worth the effort… I think it was
> Lucent labs at some point, and someone told me they generally
> don’t even have an idea about this),

Look into that DWB 3.3 link I shared.  This sounds like a very similar thing.

> so I had to bite the sour
> apple and use the last one from the Caldera drop, which is
> pretty much 1970s C code. No prototypes, and every variable
> (other than some which are short or char) is of the data type
> int or char* which are identical and interchangeable, and they
> manually paged part of the -fcommon data area relying on the
> in-memory layout to match the one from the source…

K&R C was the best language ever because it was so weakly typed. :-|

> > I hope that's a labor of love
>
> Oh, definitely!
>
> (This also allowed me to get rid of C++ from the base system;
> groff was the last remaining part, and now I can just install
> one from ports on the box where I render the ps→pdf docs.)

I regret that the implementation language makes groff so stinky for people.  Having seen the problems it was solving, I can understand why Clark selected it over the just-born ANSI C.  It's much closer to applications programming than systems programming, and C++ had much promise there.  Of course Stroustrup promoted as the best language for everything.  :-/

From today's perspective, in groff there are huge amounts of data-structure walking code that could be replaced by C++98 "algorithms" (they are, but God the name assaults the nose like a bottle of brogrammer patchouli oil), or cleanly replaced by C++11 idioms.

I get the feeling that Clark ended up not doing as much input validation as he might simply because it was so incredibly tedious to walk data structures.

Here's a recent example of some validation I added, with annotations of future possibilities.

bool is_family_valid(const char *fam)
{
  // std::vector<const char *> styles{"R", "I", "B", "BI"}; // C++11
  const size_t nstyles = 4;
  const char *st[nstyles] = { "R", "I", "B", "BI" };
  std::vector<const char *> styles(st, (st + nstyles));
  // for (auto style : styles) // C++11
  std::vector<const char *>::iterator style;
  for (style = styles.begin(); style != styles.end(); style++)
    if (!check_font(fam, *style))
      return false;
  return true;
}

C++11 would cut this function about in half.  Of course, the whole thing is pure bloat from an Annotated Reference Manual C++ perspective, where'd you skip all this ridiculous validation entirely because what could go wrong?  The ARPAnet is such a friendly place...
 

> As for tbl, IIUC the limitation is because of the limitation
> on string names in nroff.

Yeah, the two-character name space feels limited in short order.  But it made Ken Thompson's fan club ecstatic.

> I wonder if I can relax the latter
> a little in my implementation, having already raised the amount
> of things it can handle, just enough to make that page work…
>
> … gah, not tonight. No nerdsnipey for me.

Maybe I'll get you next time...

comment #11:

> The codebase is the “last” nroff I could use under the Caldera licence, i.e. that was shipped with a BSD covered by these. The macropackages are a conglomerate from the TUHS archives and pre-mdocml OpenBSD’s,

Is there anyplace these can currently be obtained?  I'm curious to see if this is the same place from which the modernized (just for build/portability purposes) DWB 3.3 troff originated.

> but I invested heavily into them to get them to mostly work (modulo missing pieces). I also added a hack (with pre‑ and postprocessor) for 8-bit/UTF-8 support, but that’s nowhere near mature.

> I only ported nroff, not troff (since nobody has the hardware it drives any more),

That wouldn't be DWB 3.3 troff, then, because people do still have PostScript devices.

https://github.com/n-t-roff/DWB3.3/tree/master/postscript/

[...]

> With the exception of terminfo(5)¹ I use this exclusively to build catmanpages, but I use groff with custom fonts to build PDF manpages for the website for the “portable subprojects” like mksh.

I hope that's a labor of love, because it sounds like punishment to me.  (Which is no doubt what some people think of some of my avocations...)

> ① “Too many text block diversions”, which probably means this cannot be rendered with tbl(1) at all and needs GNU tbl, if I understand this correctly.

Yes.

tbl(1) from groff 1.23.0:

  GNU tbl enhancements
    In addition to extensions noted above, GNU tbl removes constraints
    endured by users of AT&T tbl.

    • Region options can be specified in any lettercase.

    • There is no limit on the number of columns in a table, regardless
      of their classification, nor any limit on the number of text
      blocks.

    • All table rows are considered when deciding column widths, not
      just those occurring in the first 200 input lines of a region.
      Similarly, table continuation (.T&) tokens are recognized outside
      a region’s first 200 input lines.

    • Numeric and alphabetic entries may appear in the same column.

    • Numeric and alphabetic entries may span horizontally.

Oh sorry, submitted too early.

The codebase is the “last” nroff I could use under the Caldera licence, i.e. that was shipped with a BSD covered by these. The macropackages are a conglomerate from the TUHS archives and pre-mdocml OpenBSD’s, but I invested heavily into them to get them to mostly work (modulo missing pieces). I also added a hack (with pre‑ and postprocessor) for 8-bit/UTF-8 support, but that’s nowhere near mature.

I only ported nroff, not troff (since nobody has the hardware it drives any more), and the codebase is a mess I had cleaned up a little over the years. It builds with -O1 on ILP32 with a patched GCC 3.4.6 but GCC 4 already kills it, and sparc has an issue i386 hasn’t (or had, the latest changes might have solved that, but I have yet to have someone run electricity to where my SPARCstations are in the new house and so couldn’t test that yet).

I have recently begun working on porting it to ILP32 GNU/Linux and newer GCC so I can use UBSan to find the remaining issues, but that’s a low-priority project unfortunately.

With the exception of terminfo(5)¹ I use this exclusively to build catmanpages, but I use groff with custom fonts to build PDF manpages for the website for the “portable subprojects” like mksh.

① “Too many text block diversions”, which probably means this cannot be rendered with tbl(1) at all and needs GNU tbl, if I understand this correctly.

comment #8:

> Just chiming in that the change from GNU groff 1.22.x to 1.23 in GNU mdoc is wrapping all my manpages badly because it reduces the already very narrow screen estate.

Are the lines breaking in inappropriate places, as in the 82nd character cell on an 80-column terminal?

Or is the output "merely" ugly?

What do you mean by "very narrow"?  Over the decades man pages have "bloated" from an expected line length of 65n to more like 78n, and even then there are many carelessly composed tbl(1) tables that will overrun the line length.  In groff 1.23.0 I've made tbl diagnose that in more cases (bug #61878).

> I’m using AT&T nroff

There were many AT&T nroffs.  Which release of Unix, or DWB?

> with UCB mdoc to write my manpages, but make their rendering portable to work also with GNU groff (with BSD or GNU mdoc), and mdocml has also learnt to render them correctly.

I'm curious to know what portability issues there are!  Do you have a guide somewhere?

For instance, the 3-character request names for brace enclosures cannot possibly be portable to AT&T troff.  There is space in the mdoc macro name space to rename them such that s/Br/C/.  What do you think?

Not yet; I am hoping Bertrand will have some RC4 work for me today.

Also, I'm not settled yet on these register names.

I don't expect this change to become available until 1.23.1 or 1.24, whichever we have.

To summarize, this ticket contemplates two new registers for man(7), and restricted meaning for the `IN` register, first exposed/documented by Documenter's Workbench (DWB) 1.0 in 1984.

• BP: base paragraph indentation configures how far the left margin of a normal paragraph is inset from the page offset (which is 0 on terminals) when no relative insets are active.
• IN: the "standard indentation" would now configure only the default indentation increment used for relative inset.
• TS: the "tag spacing" configures the size the of the gap that is required after a paragraph tag.  Effectively, this amount is added to the width of the diversion created by the `IP` and `TP` macros to format the tag.  If that amount is greater than the standard indentation, a break is emitted and the remainder of the paragraph is formatted starting on a new output line, indented by the `IN` increment.

Presently, groff man(7) acts as if BP=IN=7n (7.2n on typesetters), and TS is 1n.

For consistent rendering of mdoc(7) pages, these parameters (if set) would be honored by that macro package too, as `SN` and `IN` already are, where comparable macro semantics permit.

I see no reason why the standard indentation of running text from the left margin of the paper and the default indentation inside a tagged paragraph need to be related in any way.  In fact, in 1990 Kernighan troff, the former was \n(IN and the latter was \n()I.  Both happened to be .5i, but they could be contolled independently of each other.

As another example, in mdoc(7), the default indentation inside tagged lists is 8n - 6n for the tag itself and an additional 2n for spacing, exactly as suggested by Alejandro.

Would anything be wrong with restoring the indentation of running text from the left margin from 7n to again be 5n but at the same time leaving anything that happens inside the document content, i.e. to the right of those 7n or 5n of global indentation, completely unchanged?

The setting ".nr IN 7.2n" is present since the beginning of the groff repository, i.e. groff-1.06, Sep 1 12:28:08 1992, file tmac/tmac.an .

The groff-1.01 released by James Clark on Mar 13 12:49:40 1991 and declared as a "beta-test version" that is contained in 4.3BSD-Net/2 (Aug 20, 1991) also contains ".nr IN 7.2n".

The 4.3BSD-Reno release (June 1990) does not yet contain groff (possibly because groff did not yet exist in June 1990, not even as a beta release - Wikipedia claims the first groff release was 0.3.1 in June 1990, without specifying a source) and still uses Kernighan's non-free device independent troff, licensed from AT&T.  It contains ".nr IN .5i" in the man macro set.

Consequently, it is almost certain that it was James Clark who changed from 5n to 7n during the very early stages of groff development, and definitely earlier than the 1.01 release.  In the CHANGES and ChangeLog files contained in groff-1.01, i see no explanation of why he made the change.

I also consider it likely that AT&T troff never moved away from the 5n default.  For example, UNIX v10 (1988) contains:

https://minnie.tuhs.org/cgi-bin/utree.pl?file=V10/man/man0/tmac.v10

.if n .nr )M 5n
.nr IN \\n()Mu

I admit the following remains somewhat speculative unless we ask her, but it seems likely to me that Cynthia used the 5n default for mdoc(7) because she likely did not yet have access to groff when she started mdoc(7) development: Reno already contains mdoc(7) macros, but no groff yet.  She once told me that when she got access to groff, she liked it so much that she proposed to drop support for Kernighan's troff in manual pages, but that proposal was vetoed by other members of the CSRG, so the mdoc(7) macros remained compatible with both roff implementations until the CSRG disbanded in 1995.

To summarize, it was likely James Clark who changed the indentation without so much as providing a rationale, at least not one that survives to this day, while it is mdoc(7) that upholds the original UNIX tradition in this respect.