bugGNU roff - Bugs: bug #51003, "test-groff" can't find...

 
 

bug #51003: "test-groff" can't find doc-macros in the "tmac/mdoc" directory

Submitter:  Bjarni Ingi Gislason <bjarniig>
Submitted:  Thu 11 May 2017 03:21:40 PM UTC
   
 
Category:  General Severity:  1 - Wish
Item Group:  Build/Installation Status:  Fixed
Privacy:  Public Assigned to:  gbranden
Open/Closed:  Closed Planned Release:  1.23.0
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Fri 10 Jun 2022 08:46:00 AM UTC, comment #11: 


commit de1e4091311d74ab2af19aa0d376c5d228a14280
Author: G. Branden Robinson <g.branden.robinson@gmail.com>
Date:   Tue Aug 25 17:52:55 2020 +1000

    mdoc: Relocate within build tree.

    The (modern) mdoc macro package has not been usable within the build
    tree, unlike the others.  This makes it more troublesome to test
    changes, and frustrates deploying our test infrastructure against it.
    Re-arrange the build tree to resemble an installation tree closely
    enough for the macro package to load.

            renamed:    doc-common-u -> mdoc/doc-common-u
            renamed:    doc-ditroff-u -> mdoc/doc-ditroff-u
            renamed:    doc-nroff-u -> mdoc/doc-nroff-u
            renamed:    doc-syms-u -> mdoc/doc-syms-u

    * tmac/tmac.am (TMACMDOCFILES): Look for files in tmac/mdoc.
      ($(TMACMDOCFILES)): Create tmac/mdoc in build tree.

    Fixes <https://savannah.gnu.org/bugs/index.php?51003>.


G. Branden Robinson <gbranden>
Group administrator
Tue 25 Aug 2020 08:40:30 AM UTC, comment #10: 

This has been discussed on the development list multiple times over the years, and no one objected to the idea of fixing it on principle.

There were basically two ways to fix it:

1. Make the installation tree look more like the build tree, with all the doc-* macro files in the same directory.

2. Make the build tree look more like the installation tree, with the "modules" of the mdoc package in an mdoc/ subdirectory of tmac/.

I went with option 2.  Churn for the dev team is less objectionable than churn for users, and besides I think it is a good idea to reinforce by example the notion that a macro package need not be implemented as a flat hierarchy.  Option 1 would not have done this.

Also, now this problem is in my way.  See bug #58992.

Fixed in de1e4091311d74ab2af19aa0d376c5d228a14280.

G. Branden Robinson <gbranden>
Group administrator
Thu 15 Nov 2018 12:05:16 AM UTC, comment #9: 

  My thoughts:

1) The methodology of mdoc-macros is different from the tmac-ones.
  They deserve thus a different "mother"-directory (../).
  That is used in the installed software.

2) Changes of directory names in installed software can cause problems
if some local files rely on the names being invariant.

3) The problem that "test-groff" poses, is that a different
"mother-directory" name is used for the repository, build-directory
and the installed one.

Conclusion:

  Use the same name for the "mother-directory" in all places:
repository, build-directory, tar-file, and the installation.

  So the task would be:

1) to create a tmac/mdoc directory in the repository and move the
(non-tmac) "mdoc-*" files" (the four unstripped ones) to that one

2) adjust the *.am files to the changes.

Bjarni Ingi Gislason <bjarniig>
Wed 14 Nov 2018 03:52:45 PM UTC, comment #8: 

gbranden@ re comment #7:

Yes, git master HEAD still installs the four doc-* macro files into tmac/mdoc, that didn't change.

I wouldn't object to installing them directly into the main tmac/ macro directory as tmac/doc-*.tmac - it's not that polluting, after all, it's only four files.  Of course, that requires changing the four .mso statements in doc.tmac at the same time.

Ingo Schwarze <schwarze>
Group Member
Wed 14 Nov 2018 03:27:20 PM UTC, comment #7: 

In groff 1.22.3, a subdirectory of tmac called mdoc is shipped and populated with the files that are being sourced:


$ dpkg -L groff-base|grep 'tmac.*doc'
/usr/share/groff/1.22.3/tmac/andoc.tmac
/usr/share/groff/1.22.3/tmac/doc-old.tmac
/usr/share/groff/1.22.3/tmac/doc.tmac
/usr/share/groff/1.22.3/tmac/mandoc.tmac
/usr/share/groff/1.22.3/tmac/mdoc
/usr/share/groff/1.22.3/tmac/mdoc/doc-common
/usr/share/groff/1.22.3/tmac/mdoc/doc-ditroff
/usr/share/groff/1.22.3/tmac/mdoc/doc-nroff
/usr/share/groff/1.22.3/tmac/mdoc/doc-syms
/usr/share/groff/1.22.3/tmac/mdoc.tmac


I'll see if we're still doing that.  I'll also play around with GROFF_TMAC_PATH to see if I can get those .mso requests to complain when they should be.

I'm not giving up on this yet because I really want to be able to A/B changes to groff_mdoc (or the macro package itself, for that matter) in built trees without having to do an install.

Without the patch, that's totally unreasonable.  If you run the commands in my earlier comments, you get vast quantities of unholy spew.


$ groff -t -M build/tmac -mdoc build/tmac/groff_mdoc.7 2>&1 >/dev/null | head
Usage: .Sh section_name ... (#115)
Usage: .Sh section_name ... (#245)
Usage: .Ss not callable by other macros (#261)
Usage: .Li argument ... (#313)
Usage: .Li argument ... (#316)
Usage: .Li argument ... (#320)
Usage: .Li argument ... (#322)
build/tmac/groff_mdoc.7:325: warning: macro `doc-first-parameter' not defined (possibly missing space after `do')
build/tmac/groff_mdoc.7:325: warning: macro `doc-get-width' not defined (possibly missing space after `do')
build/tmac/groff_mdoc.7:325: warning: macro `doc-get-arg-width' not defined (possibly missing space after `do')

[literally ONE THOUSAND MORE LINES of stderr snipped]

I may end up agreeing with you, but I'm not willing to admit defeat yet.

G. Branden Robinson <gbranden>
Group administrator
Wed 14 Nov 2018 08:03:05 AM UTC, comment #6: 

gbranden@, re comment #5 and file #45421:

I think your proposed patch is very wrong.  If i apply that to my installed version of doc.tmac, it totally breaks formatting of manual pages with groff:

  schwarze@isnote $ groff -mandoc -Tascii -P -c /usr/share/man/man1/cat.1
  troff: /usr/local/share/groff/1.22.4/tmac/doc.tmac:58: warning:     can't find macro file 'doc-nroff'
  troff: /usr/local/share/groff/1.22.4/tmac/doc.tmac:59: warning: can't find macro file 'doc-common'
  troff: /usr/local/share/groff/1.22.4/tmac/doc.tmac:60: warning: can't find macro file 'doc-syms'
  troff: /usr/share/man/man1/cat.1:62: warning: macro 'doc-get-arg-width' not defined (possibly missing space after 'do')
  Usage: .Ex -std in sections 1, 6 and 8 only (#99)
  cat cat Flbenstuv Ar The cat utility reads files sequentially,
  writing them to the standard output.  The file operands are
  [...]

I still don't think there is any problem to solve.  Just close this ticket as invalid.

Ingo Schwarze <schwarze>
Group Member
Wed 14 Nov 2018 04:46:10 AM UTC, comment #5: 

I think actually Bjarni was right here.  Scrubbing those pointless mdoc/ prefixes from the arguments to the .mso calls makes it straightforward to just run groff (even a system groff) on the built groff_mdoc.7 with the macros from the build.

This works for out-of-tree builds:


groff -t -M build/tmac -mdoc build/tmac/groff_mdoc.7 2>&1 >/dev/null


And in-tree ones:


groff -t -M tmac -mdoc tmac/groff_mdoc.7 2>&1 >/dev/null


This happens to fix the test-groff case with no extra effort.


./build/test-groff -t -mdoc ./build/tmac/groff_mdoc.7 2>&1 > /dev/null # out of tree
./test-groff -t -mdoc ./tmac/groff_mdoc.7 2>&1 > /dev/null # in tree


Now, the question that really animates me is why .mso wasn't screaming to stderr when it couldn't find mdoc/whatever.  It's supposed to; I was just hacking on input.cpp:macro_source() the other day.  But that's a separate bug.

Proposed fix attached.  Ingo, please comment if you have a moment.


(file #45421)

G. Branden Robinson <gbranden>
Group administrator
Sat 13 May 2017 03:10:50 PM UTC, comment #4: 


> It could (to whom?, and why?. Is it solvable?).


To developers trying to maintain groff, and to users trying to install it from source, trying to port it to new platforms, and trying to debug problems they encounter.  Because build systems are already complicated without adding needless complication.

It is not completely solvable.  A build system of software as complex as groff will always remain complicated.  The best we can do is keep it as simple as possible.

It is NOT the job of a build system to try and create a half-working pseudo-installation inside the source directory.  The job of the source tree is to contain one copy of each required source file in a place that is logical for development (and not as an installation location).  The job of the build system is to read the source files and create one copy of each required autogenerated file, in whichever location in the build directory is simplest.  The job of the install target is to copy these autogenerated files (and maybe some source files) to as many installation locations as required for the software to work.  Mixing up these jobs is a recipe for confusion and bugs.

> Creating the missing directory and the links in the
> git-repository was for me the simplest solution.


It doesn't even work reliably.  If you follow the advice in INSTALL.REPO to create a separate build directory, it will fail.

> Another is to let the Makefile create the directory and links.


Yeah, somehere is tmac/tmac.am or somewhere similar, that might work.  But it's likely to remain fragile, and it will definitely add additional complexity to the build system.  We should strive to simplify the build system, not make it even more complicated than it already is.

Really, hacking around in order to make test-groff better is not worth making maintenance harder and increasing the risk of bugs in the released software.

Ingo Schwarze <schwarze>
Group Member
Sat 13 May 2017 01:10:18 AM UTC, comment #3: 


> Follow-up Comment #1, bug #51003 (project groff):
>
> I suspect this ticket to be invalid.
>


  A suspicion is a guess.  I have not seen any (valid) evidence for it to
be true.

> > Man-pages include "doc"-macros with a request ".mso mdoc/...".
>
> No.  They don't.  In the complete OpenBSD ports tree (nearly 10k ports), i'm
> not aware of a single port doing that.
>


  You are right in a way.  Clumsy wording on my part.  Better is
'"doc"-macros are included with a request ".mso mdoc/..." if needed."
I was pointing out how the directory "mdoc" is involved in the inability of
"test-groff" to function correctly.

[...]

> > A fix could be
>
> A fix for what exactly?  Which problem are you trying to solve?
>


  My subject line shows that.
  And a solution would also get rid of the following in the "PROBLEMS" file:

"* Processing mdoc man pages like `groff_mdoc.n' fails if I use
  `test-groff'.  Why?

The mdoc package works only after installation, not with test-groff.
Reason is that the macro files of mdoc get stripped off the `doc-'
prefix before they are copied to the final destination."

> > to create the directory and links in that directory
>
> Certainly not.  There is no need whatsoever to have duplicate versions of
> files in the source tree.  That would only cause confusion.
>


  It could (to whom?, and why?.  Is it solvable?).  Creating the missing
directory and the links in the git-repository was for me the simplest
solution.
Another is to let the Makefile create the directory and links. And adding
"tmac/mdoc" to the ".gitignore" file.  See for example "build-aux".

[...]

Bjarni Ingi Gislason <bjarniig>
Thu 11 May 2017 08:45:15 PM UTC, comment #2: 

Oops, sorry for missing the first word of the bug title.  But i still think there is no problem.  I have often worked on bugfixing in the mdoc macros, but a need to test them with ./test-groff never came up.  "Fixing" that isn't worth any complication.

Ingo Schwarze <schwarze>
Group Member
Thu 11 May 2017 08:13:40 PM UTC, comment #1: 

I suspect this ticket to be invalid.

> Man-pages include "doc"-macros with a request ".mso mdoc/...".


No.  They don't.  In the complete OpenBSD ports tree (nearly 10k ports), i'm not aware of a single port doing that.

If any manual page would do that, it would be extremely bad style, not serving any purpose whatsoever but introducing pointless fragility.  No manual page should ever contain any .mso request.

The way to format a manual page is calling "nroff -mandoc" (or troff -mandoc).  If you are sure it is mdoc(7) and not man(7), you can also say "nroff -mdoc", but there is no real reason why the user should bother.  The page itself certainly must not attempt to second-guess the system roff installation.

> The directory "mdoc" does not exist in the git-repository.


True.  But for one thing, source directory layout need not match the layout of the installed software.  Here, a directory like ".../groff/1.22.3/tmac/mdoc/" does get created at install time, see tmac/tmac.am, variable $(mdocdir) for details.

Then, it doesn't matter.  All that is called externally - and not by the end user or the manpage itself, but by man(1)! - is the main macro file, ../groff/1.22.3/tmac/doc.tmac, which is not inside the mdoc/ subdir.  That file includes whatever it needs.

> A fix could be


A fix for what exactly?  Which problem are you trying to solve?

> to create the directory and links in that directory


Certainly not.  There is no need whatsoever to have duplicate versions of files in the source tree.  That would only cause confusion.

> A comment in "doc.tmac":


Admittedly, that comment is ridiculous nowadays.  If worrying about systems unable to handle filenames longer than 8+3 bytes ever made sense (which i doubt), those times are over for at least two decades.  But i don't see much benefit in renaming stuff, the current names work and people got used to them.  Maybe the comment could be changed to say "nonstandard naming for historical reasons", but that doesn't seem very pressing either.

Ingo Schwarze <schwarze>
Group Member
Thu 11 May 2017 03:21:40 PM UTC, original submission:  

  Man-pages include "doc"-macros with a request ".mso mdoc/...".

  The directory "mdoc" does not exist in the git-repository.

  A fix could be to create the directory and links in that directory
to the "doc-" files (doc-common, doc-ditroff, doc-nroff, and doc-syms).

  A comment in "doc.tmac":

.\" Modified by -email is unavailable- as follows: the doc-* files are assumed to be
.\" installed as mdoc/doc-* rather than tmac.doc-* (the filename
.\" `tmac.doc-common' would be too long); when using groff, the doc-* files
.\" are loaded using the `mso' request.


Bjarni Ingi Gislason <bjarniig>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #45421:  51003.diff added by gbranden (764B - text/x-patch)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by gbranden (Updated the item)
  • -email is unavailable- added by schwarze (Posted a comment)
  • -email is unavailable- added by bjarniig (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 10 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2022-06-15 gbranden CategoryNone General
    2020-08-25 gbranden StatusIn Progress Fixed
    2020-08-25 gbranden Open/ClosedOpen Closed
        Planned ReleaseNone 1.23.0
    2018-11-14 gbranden StatusNeed Info In Progress
    2018-11-14 gbranden Attached File- Added 51003.diff, #45421
        StatusNone Need Info
        Assigned toNone gbranden
    2018-02-16 schwarze Severity3 - Normal 1 - Wish
        Item GroupNone Build/Installation

    Back to the top

    Powered by Savane 3.13-4448.
    Corresponding source code