bugGNU roff - Bugs: bug #62774, [mdoc] cope with degenerate...

 
 

bug #62774: [mdoc] cope with degenerate documents (`Dd`, `Dt`, `Os` not called)

Submitter:  G. Branden Robinson <gbranden>
Submitted:  Sat 16 Jul 2022 10:46:19 AM UTC
   
 
Category:  Macro mdoc Severity:  3 - Normal
Item Group:  Warning/Suspicious behaviour Status:  Fixed
Privacy:  Public Assigned to:  gbranden
Open/Closed:  Closed Planned Release:  1.23.0
* Mandatory Fields

Add a New Comment Rich Markup
   

Sun 11 Sep 2022 12:23:54 PM UTC, comment #4: 


commit 4080ed7f3b4a3c7e361dec2cb4276a2d6398bd89
Author: G. Branden Robinson <g.branden.robinson@gmail.com>
Date:   Tue Sep 6 18:38:33 2022 -0500

    [mdoc]: Handle setup macro use degeneracy better.

    * tmac/mdoc/doc-common: Populate default date string `doc-date-string`
      with "UNDATED".

      (Dd, Dt, Os): Make more orthogonal: stop touching `doc-topic-name`.
      It is "doc.tmac"'s job to initialize it and the `Nm` macro's job to
      update it.

      (Dd): Do nothing if given no arguments.  (You can still blank
      out the document date with an explicitly empty argument:
        .Dd ""
      .)

    Fixes <https://savannah.gnu.org/bugs/?62774> (1/7).

    Fix code style nit: end string defs with comment escape sequences.

commit 16fe833f0ae094516aafd4c1b591f21d3a4ba112
Author: G. Branden Robinson <g.branden.robinson@gmail.com>
Date:   Tue Sep 6 18:57:04 2022 -0500

    [mdoc]: Relieve `Dt` of reinitialization duty.

    * tmac/mdoc/doc-common (Dt): Stop reinitializing `doc-volume` and
      `doc-section`.  This is the job of package initialization (and, soon,
      the end-of-input macro).

    Fixes <https://savannah.gnu.org/bugs/?62774> (2/7).

commit 1db6f86623c7f4bb5271fa5c5e4f1a2e47e030cd
Author: G. Branden Robinson <g.branden.robinson@gmail.com>
Date:   Tue Sep 6 19:36:16 2022 -0500

    [mdoc]: Reset strings between documents.

    * tmac/mdoc/doc-common (doc-end-macro): Reset strings to reduce info
      leaks from one man page to the next.

    Fixes <https://savannah.gnu.org/bugs/?62774> (3/7).

commit a241f55c920b45fcb42f7b8a33bc7a2bbd8f6722
Author: G. Branden Robinson <g.branden.robinson@gmail.com>
Date:   Tue Sep 6 20:32:05 2022 -0500

    [mdoc]: Manage footer formatting more coherently.

    * tmac/mdoc/doc-common (doc-footer): Pull more footer concerns into this
      macro.  Don't bail out if continuously rendering; instead, format the
      appropriate footer text if we are.  For now, break the page only if we
      are not continuously rendering.

      (doc-end-macro): Call `doc-footer` instead of fomatting footer here.

    Fixes <https://savannah.gnu.org/bugs/?62774> (4/7).

    Indentation is _not_ corrected in this commit, for clarity.

commit da5ad142d88d167dd57b722d96b1ca0b2a0bbbbf
Author: G. Branden Robinson <g.branden.robinson@gmail.com>
Date:   Tue Sep 6 23:03:13 2022 -0500

    [mdoc]: Warn if Dd, Dt called without arguments.

    * tmac/mdoc/doc-common (Dt, Dd): Emit diagnostic warnings if macros
      called with no arguments.

    Fixes <https://savannah.gnu.org/bugs/?62774> (5/7).

    I dislike this diagnostic message format, but it's consistent with
    existing groff mdoc practice.

commit 892703b17e088be214c8d0d91db14d2bd6a8dcd5
Author: G. Branden Robinson <g.branden.robinson@gmail.com>
Date:   Tue Sep 6 23:47:40 2022 -0500

    [mdoc]: Align footer management closer to man(7).

    * tmac/doc.tmac (doc-save-global-vars, doc-restore-global-vars):
    * tmac/mdoc/doc-common (Rd):
    * tmac/mdoc/doc-ditroff:
    * tmac/mdoc/doc-nroff: Stop defining, saving, and restoring
      `doc-footer-space`.

    * tmac/mdoc/doc-common (doc-break-body-text): New macro, called only
      when not continuously rendering, schedules a page break.

      (doc-footer): Stop vertically spacing by `doc-footer-space`, instead
      relying upon a trap to move us here.

    * tmac/mdoc/doc-ditroff (doc-setup-header):
    * tmac/mdoc/doc-nroff (doc-setup-header): Set traps at the end of the
      page where man(7) does by default; break the body text at
      1 inch from the page bottom, and write the footer at ½ inch.

    * tmac/mdoc/doc-nroff (doc-setup-header): Add 1/6 inch "thin space"
      between man page name and parenthesized section, for precise parity
      with doc-ditroff implementation.  (`\|` does not take up any character
      cells on nroff devices.)

    Fixes <https://savannah.gnu.org/bugs/?62774> (6/7).

commit 50a2d4165f7b82cb78d7ee96484f776c11a47def
Author: G. Branden Robinson <g.branden.robinson@gmail.com>
Date:   Wed Sep 7 20:00:31 2022 -0500

    [mdoc]: Align trap mgmt more closely with man(7).

    * tmac/mdoc/doc-common: Call new macro (see below)
      `doc-set-up-continuous-rendering` when initializing, if that is mode
      is configured.

      (doc-ne): New macro replaces `ne` request in continuous rendering
      mode.  It extends the page length by the amount in the argument or by
      1v if none given.

      (doc-bp): New macro replaces `bp` request in continuous rendering
      mode, setting the page length to the vertical drawing position.

      (doc-set-up-continuous-rendering): New macro renames requests to
      emplace the foregoing.

      (Dd): Interpret this macro call strictly as starting a new mdoc(7)
      document.  (andoc.tmac already makes this assumption, and has for over
      20 years.  groff_mdoc(7) and mandoc_mdoc(7) also prescribe the
      sequence `Dd`, `Dt`, `Os`.)  We require this invariant even more
      rigidly now because it's the only way we can be sure that we can
      process multiple documents while rendering headers and footers with
      information corresponding to the appropriate document.  (man(7)'s `TH`
      has an advantage here in that calling it is "atomic": from its
      arguments alone you can obtain everything you need to know to format
      the header and footer.  In mdoc(7), permuting the initialization macro
      order reliably produces chaos.)  Break the page (if necessary)
      _before_ processing any arguments (instead of after), to flush the
      previous page's footer.  Stop calling `doc-set-up-titles` here; we
      don't have enough information to do that yet.  Also stop writing the
      PDF bookmark here, because `doc-document-title` and `doc-section` will
      not reflect the new page content yet.

      (Os): Once the `doc-operating-system` string content has been
      determined, call `doc-set-up-titles`, write the PDF bookmark for the
      page, and call `doc-header`, causing the page header to be formatted.
      These changes further imply a stronger requirement on initialization
      macro ordering being canonical.

    Fixes <https://savannah.gnu.org/bugs/?62774> (7/7).


G. Branden Robinson <gbranden>
Group administrator
Sun 11 Sep 2022 12:09:58 PM UTC, comment #3: 

I should add there's no regression relative to groff 1.22.4.  Ingo terms every change in output from his test suite for mandoc as a "regression", which I find overbroad and unhelpful.

groff 1.22.4 indeed behaved badly with degenerate input.  Here's an exhibit.



$ cat EXPERIMENTS/degenerate.man
.Dd
.Dt
.Os
.Dd Today
.Dt foo 1
.Os Hurd
.Sh Name
.Nm gasguzzler
.Dd Tomorrow
.Dt bar 3
.Os
$ groff --version | head -n 1
GNU groff version 1.22.4
$ groff -Tascii -mdoc EXPERIMENTS/degenerate.man | cat -s
mdoc warning: Empty input line #34

Name
     gasguzzler


The foregoing input also badly confused continuous rendering mode, spraying many blank lines to the terminal, hence the "cat -s".


$ groff -Tascii -mdoc EXPERIMENTS/degenerate.man | wc -l
mdoc warning: Empty input line #34
66


With the changes I'm about to push, we have the following instead.


$ ./build/test-groff -b -ww -Tascii -mdoc EXPERIMENTS/degenerate.man
mdoc warning: .Dd directive expects an argument (#1)
mdoc warning: .Dt directive expects one or more arguments (#2)
UNTITLED                             LOCAL                            UNTITLED

GNU                                 UNDATED                           UNTITLED
-------------------------------------------------------------------------------
foo(1)                      General Commands Manual                     foo(1)

Name
     gasguzzler

Hurd                                 Today                              foo(1)
-------------------------------------------------------------------------------
bar(3)                      Library Functions Manual                    bar(3)

GNU                                Tomorrow                             bar(3)


Note that the mail gateway or an MUA may corrupt the long, whitespace-heavy lines above.  The headers and footers are one line each.

I can't speak for Ingo, but this looks like progress to me.

G. Branden Robinson <gbranden>
Group administrator
Sun 11 Sep 2022 08:00:25 AM UTC, comment #2: 

As with bug #62773, and having looked at the practicalities, I think our mdoc package need only deal with degenerate documents on a best-effort basis.

That said, there are definitely some lingering bugs involving page trap management, particularly visible when batch rendering of multiple mdoc documents was attempting.

So I've addressed those across several commits in my working copy.

And added some warnings when `Dd` and `Dt` are called with insufficient arguments.

...and aligned the header and footer layouts with man(7).


commit 0db8cce06529fbe5c80f8d19c1002ea5d5d7917d
Author: G. Branden Robinson <g.branden.robinson@gmail.com>
Date:   Tue Sep 6 18:38:33 2022 -0500

    [mdoc]: Handle setup macro use degeneracy better.

    * tmac/mdoc/doc-common: Populate default date string `doc-date-string`
      with "UNDATED".

      (Dd, Dt, Os): Make more orthogonal: stop touching `doc-topic-name`.
      It is "doc.tmac"'s job to initialize it and the `Nm` macro's job to
      update it.

      (Dd): Do nothing if given no arguments.  (You can still blank
      out the document date with an explicitly empty argument:
        .Dd ""
      .)

    Fix code style nit: end string defs with comment escape sequences.

commit 15a8b468b6f3b3769f45b9d1f00e4bec2537619e
Author: G. Branden Robinson <g.branden.robinson@gmail.com>
Date:   Tue Sep 6 18:57:04 2022 -0500

    [mdoc]: Relieve `Dt` of reinitialization duty.

    * tmac/mdoc/doc-common (Dt): Stop reinitializing `doc-volume` and
      `doc-section`.  This is the job of package initialization (and, soon,
      the end-of-input macro).

commit a903b34b44a62b0b392ed87088f4d6e37f8f71a4
Author: G. Branden Robinson <g.branden.robinson@gmail.com>
Date:   Tue Sep 6 19:36:16 2022 -0500

    [mdoc]: Reset strings between documents.

    * tmac/mdoc/doc-common (doc-end-macro): Reset strings to reduce info
      leaks from one man page to the next.

commit 98c2ab319477b67882073293c7ed98cda01a2428
Author: G. Branden Robinson <g.branden.robinson@gmail.com>
Date:   Tue Sep 6 20:32:05 2022 -0500

    [mdoc]: Manage footer formatting more coherently.

    * tmac/mdoc/doc-common (doc-footer): Pull more footer concerns into this
      macro.  Don't bail out if continuously rendering; instead, format the
      appropriate footer text if we are.  For now, break the page only if we
      are not continuously rendering.

      (doc-end-macro): Call `doc-footer` instead of fomatting footer here.

commit b483876a5937712334f88f5580e98f2bc2cd70f8
Author: G. Branden Robinson <g.branden.robinson@gmail.com>
Date:   Tue Sep 6 23:03:13 2022 -0500

    [mdoc]: Warn if Dd, Dt called without arguments.

    * tmac/mdoc/doc-common (Dt, Dd): Emit diagnostic warnings if macros
      called with no arguments.

    I dislike this diagnostic message format, but it's consistent with
    existing groff mdoc practice.

commit 0335372b8c0d1628b5874490b8b55e8f8d2c0d19
Author: G. Branden Robinson <g.branden.robinson@gmail.com>
Date:   Tue Sep 6 23:47:40 2022 -0500

    [mdoc]: Align footer management closer to man(7).

    * tmac/doc.tmac (doc-save-global-vars, doc-restore-global-vars):
    * tmac/mdoc/doc-common (Rd):
    * tmac/mdoc/doc-ditroff:
    * tmac/mdoc/doc-nroff: Stop defining, saving, and restoring
      `doc-footer-space`.

    * tmac/mdoc/doc-common (doc-break-body-text): New macro, called only
      when not continuously rendering, schedules a page break.

      (doc-footer): Stop vertically spacing by `doc-footer-space`, instead
      relying upon a trap to move us here.

    * tmac/mdoc/doc-ditroff (doc-setup-header):
    * tmac/mdoc/doc-nroff (doc-setup-header): Set traps at the end of the
      page where man(7) does by default; break the body text at
      1 inch from the page bottom, and write the footer at ½ inch.

    * tmac/mdoc/doc-nroff (doc-setup-header): Add 1/6 inch "thin space"
      between man page name and parenthesized section, for precise parity
      with doc-ditroff implementation.  (`\|` does not take up any character
      cells on nroff devices.)

commit e7acc0dd49bf472d7bdc2da7d726df58bd7c8ad9
Author: G. Branden Robinson <g.branden.robinson@gmail.com>
Date:   Wed Sep 7 20:00:31 2022 -0500

    [mdoc]: Align trap mgmt more closely with man(7).

    * tmac/mdoc/doc-common: Call new macro (see below)
      `doc-set-up-continuous-rendering` when initializing, if that is mode
      is configured.

      (doc-ne): New macro replaces `ne` request in continuous rendering
      mode.  It extends the page length by the amount in the argument or by
      1v if none given.

      (doc-bp): New macro replaces `bp` request in continuous rendering
      mode, setting the page length to the vertical drawing position.

      (doc-set-up-continuous-rendering): New macro renames requests to
      emplace the foregoing.

      (Dd): Interpret this macro call strictly as starting a new mdoc(7)
      document.  (andoc.tmac already makes this assumption, and has for over
      20 years.  groff_mdoc(7) and mandoc_mdoc(7) also prescribe the
      sequence `Dd`, `Dt`, `Os`.)  We require this invariant even more
      rigidly now because it's the only way we can be sure that we can
      process multiple documents while rendering headers and footers with
      information corresponding to the appropriate document.  (man(7)'s `TH`
      has an advantage here in that calling it is "atomic": from its
      arguments alone you can obtain everything you need to know to format
      the header and footer.  In mdoc(7), permuting the initialization macro
      order reliably produces chaos.)  Break the page (if necessary)
      _before_ processing any arguments (instead of after), to flush the
      previous page's footer.  Stop calling `doc-set-up-titles` here; we
      don't have enough information to do that yet.  Also stop writing the
      PDF bookmark here, because `doc-document-title` and `doc-section` will
      not reflect the new page content yet.

      (Os): Once the `doc-operating-system` string content has been
      determined, call `doc-set-up-titles`, write the PDF bookmark for the
      page, and call `doc-header`, causing the page header to be formatted.
      These changes further imply a stronger requirement on initialization
      macro ordering being canonical.

commit e2c94c33b0d83714c70e62766e400edbf7d4ebe8
Author: G. Branden Robinson <g.branden.robinson@gmail.com>
Date:   Wed Sep 7 20:55:36 2022 -0500

    [mdoc]: Align spacing around headers with man(7).

    * tmac/doc.tmac (doc-save-global-vars, doc-restore-global-vars):
    * tmac/mdoc/doc-common (Rd):
    * tmac/mdoc/doc-ditroff:
    * tmac/mdoc/doc-nroff: Stop defining, saving, and restoring
      `doc-header-space`.

    * tmac/mdoc/doc-common (doc-header): When starting new document and not
      continuously rendering, space by one half-inch instead of by the
      now-defunct `doc-header-space` amount (which, strictly, should not
      have applied to this measurement).  If not continuously rendering,
      extend page length by 1v to accommodate header line.  After formatting
      header, if not continuously rendering, space to 1 inch below first
      text baseline.

commit 15a0e870772ec3102c24db495cc501c926c6ade8
Author: G. Branden Robinson <g.branden.robinson@gmail.com>
Date:   Wed Sep 7 23:29:29 2022 -0500

    [mdoc]: Stop prefixing center header with "BSD".

    Stop prefixing center header with name of operating system.  This was
    hard-coded to "BSD" and stuck on all mdoc(7) pages using a default
    manual section number, regardless of host operating system.  Bad idea.
    Further, the argument to the `Os` call, if any, or a default, is always
    disclosed in the left footer, so the foregoing practice gave the reader
    no new information.  (On the other hand, header and footer rendering was
    not reliable in continuous rendering mode, the default for terminals, in
    groff 1.22.4 and earlier.  It is now.)

    * tmac/mdoc/doc-common: Do it.

      (Dt): Initialize `doc-volume` as empty and annotate it; calling the
      macro with a numeric second argument overrides the fallback default of
      "LOCAL".  A slash is no longer needed to separate a recognized
      architecture string from the OS name, so drop it.

    * tmac/groff_mdoc.7.man: Stop using the string in examples.  Also
      correct table format specification declaring 3 columns but using 2.

    * tmac/tests/doc_smoke-test.sh: Update test expectations.


G. Branden Robinson <gbranden>
Group administrator
Tue 23 Aug 2022 09:33:44 PM UTC, comment #1: 

Kicking up to blocker severity because this needs to be done before 1.23.0 finalizes.

This is going to be less straightforward than it looks because of batch rendering.  (Just because the info is set doesn't mean it's correct for you page--maybe the previous man page set it.)  Might need to reset strings to dummy values in the end-of-input macro as well as when initializing the package.  Try to use as closely parallel an approach as possible in tmac/an.tmac.

G. Branden Robinson <gbranden>
Group administrator
Sat 16 Jul 2022 10:46:19 AM UTC, original submission:  

Ingo Schwarze reported several issues with malformed mdoc(7) documents to the mailing list.

https://lists.gnu.org/archive/html/groff/2022-06/msg00117.html

Here is an edited version of my response to his report.

> I'll have a fresh look at my changes to groff mdoc in this commit and
> see if I can find good spots to recover more gracefully.


> I have to admit I'm tempted to either let these fail, or, if I can find
> a good place to stick a sanity-checking hook, simply refuse to render
> the page if these macros, documented as mandatory, are missing.  But,
> maybe replacing the missing content with shouty-caps stuff like
> "UNTITLED" and "LOCAL" suffices to clue the user in.  Perhaps also
> changing the fallback operating system string from "BSD" to "GNU" will
> more effectively agitate misusers of mdoc(7) into correcting their ways.


> (Relatedly, I don't understand why anyone thought it was a good idea for
> the volume titles for mdoc(7) man pages in a GNU project to all announce
> themselves as being from a BSD manual even if they're rendering man
> pages that have nothing to do with BSD.  This name is not derived from
> any argument to the `Os` macro, nor configured based on the build host's
> OS identity, but hard-coded in `doc-volume-operating-system`.  If this
> string were made empty or deleted, the volume titles would exactly match
> those used by groff man(7).)


Right now my idea is to move some of the string resets from the `Dt` to the `Dd` macro definition.

Work up some regression tests for this.

Also look into killing off `doc-volume-operating-system`, noted above.

G. Branden Robinson <gbranden>
Group administrator

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by gbranden (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 8 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2022-09-11 gbranden StatusIn Progress Fixed
        Assigned toNone gbranden
        Open/ClosedOpen Closed
    2022-09-11 gbranden Planned ReleaseNone 1.23.0
    2022-09-11 gbranden Severity5 - Blocker 3 - Normal
        StatusNone In Progress
        Summary[mdoc] warn if any of `Dd`, `Dt`, `Os` not called [mdoc] cope with degenerate documents (`Dd`, `Dt`, `Os` not called)
    2022-08-23 gbranden Severity3 - Normal 5 - Blocker

    Back to the top

    Powered by Savane 3.13-d3ae.
    Corresponding source code