bugGNU troff - Bugs: bug #60602, Unit discrepancy in \r, \u, and \d...

 
 

bug #60602: Unit discrepancy in \r, \u, and \d between Texinfo manual and groff(7)

Submitted by:  Dave <barx>
Submitted on:  Thu 13 May 2021 07:45:39 PM UTC
 
Category:  Core Severity:  2 - Minor
Item Group:  Documentation Status:  Invalid
Privacy:  Public Assigned to:  gbranden
Open/Closed:  Closed Planned Release:  None
* Mandatory Fields

Add a New Comment (Rich Markup)
   

Comment Type & Canned Response:
       No canned response available

 

( Jump to the original submission )

Tue 23 Nov 2021 03:13:39 PM UTC, comment #16: 

This turned out to be invalid; it's been reverted and its ChangeLog entry deleted.

Also our Texinfo manual is now fixed.

commit c709a70a6b2bb2bbeb7991c4e8dff7cd7e7c9031
Author: G. Branden Robinson <g.branden.robinson@gmail.com>
Date:   Tue Nov 23 23:57:51 2021 +1100

    [docs]: Fix unit error in `\[dru]` descriptions.

    * doc/groff.texi (Page Motions): Fix error: the `\r`, `\u`, and `\d`
      escape sequences move in ems, not vees, despite being vertical
      motions.  Add discussion and example.

    Problem dates back to import of Texinfo manual, commit af2d6f8e (26
    December 1999).

    Also reverts commit c6a01c81 by me, 20 May 2021.

G. Branden Robinson <gbranden>
Project Administrator
Tue 23 Nov 2021 05:31:24 AM UTC, comment #15: 

> In fact, a C version of cat2dit that would run on V7 Unix under SIMH would, in my imagination, kick ass.  (People really wanted one, back in the '80s.)  Then there would be no data migration issue or octal dumping to mess with.  Run SIMH in a terminal window with enough scrollback and you could just copy and paste the entire ditroff output to wherever you wanted.


I've an even better idea: implement a driver for SIMH's PDP-11 emulator that advertises itself as a connected C/A/T device. It's possible to emulate paper-tape rolls and even Ethernet adapters for a SIMH client, but that's well above my paygrade:

http://simh.trailing-edge.com/pdf/pdp11_doc.pdf

I imagine most of the terminology in this document is less opaque to you than it is to me (Hell, I can barely write a C program to save my life). In any case, if you still want a non-JS implementation of cat2dit, the source I got it from was a C header file with all the macros provided and ready to go. They even provided a man page!

The original UseNet post was
https://groups.google.com/d/topic/comp.text/IFbbkuI91nA

John Gardner <alhadis>
Tue 23 Nov 2021 05:13:36 AM UTC, comment #14: 

> this GitHub project seems not to have been updated in a while (December 2019, IIRC), and Node itself has moved on.


That's not how things work in the JavaScript world. ECMAScript, unlike Python, is designed to remain compatible with all existing and legacy codebases, as introducing any incompatibilities would break existing websites.

What you might be seeing is an uncaught exception raised by Roff.js's attempt at loading a tmac file that expects ms(7) to be loaded first (it appears to be a new package, as I would've noticed `.ab` being used and fixed it a lot earlier…).

Roff.js isn't reliant on Node, though, and though it should theoretically work in the browser console without modification… except I only just realised how slack I was with this project, as  running this in the online-demo shows a missing function:

view.loadFile("https://github.com/Alhadis/Roff.js/raw/master/test/fixtures/cat/v7-ed.cat");
ReferenceError: readFile is not defined
columnNumber: 32211
fileName: "https://rawgit.com/Alhadis/Roff.js/web-demo/min/all.js"​
lineNumber: 1
message: "readFile is not defined"

I really need to resume work on this, because it's still very much an unfinished project. You otherwise could've pasted the raw C/A/T output into the previewer and it would've interpreted it properly (provided it was transmitted as binary).

I'm an idiot...

John Gardner <alhadis>
Tue 23 Nov 2021 02:16:09 AM UTC, comment #13: 

comment #10:

> CSTR 54 is correct.  Branden's 'guess' is wrong and no change from Bell Labs troff's normal and documented behaviour should be made based on it.  vflag is used to indicate the motion is vertical.  It is nothing to do with forcing a unit calculation to be vees.  The code of makem(), quoted but not analysed above, makes this clear.


   443  makem(i)
   444  int i;
   445  {
   446          register j;
   447
   448          if((j = i) < 0)j = -j;
   449          j = (j & ~MOTV) | MOT;
   450          if(i < 0)j |= NMOT;
   451          if(vflag)j |= VMOT;
   452          return(j);
   453  }

https://www.youtube.com/watch?v=KmKOVdAGtzM

G. Branden Robinson <gbranden>
Project Administrator
Tue 23 Nov 2021 02:08:22 AM UTC, comment #12: 

Hi John,

I had this sitting in a browser text box for days, so I'm just going ahead and pushing it out to record the state of things.

comment #4:

> > because I have nothing that can interpret the Graphic Systems C/A/T command stream the former produces.
>
> I do:
>

> $ cat2dit -Tps   < output.cat | grops
> $ cat2dit -Tpost < output.cat | dpost
>
> # Unscaled translation of C/A/T output
> $ cat2dit < output.cat

>
> Based on https://github.com/Alhadis/otroff/tree/cat-driver
>
> https://github.com/Alhadis/otroff/tree/cat-driver


Unfortunately, I hit more roadblocks...this GitHub project seems not to have been updated in a while (December 2019, IIRC), and Node itself has moved on.  So my Debian buster-based system is too old for your code, and the macOS system I sometimes have access to is too new for it.  I don't have the error messages handy but I can collect them if that would be helpful.  They're dependency issues, probably exactly the sort of thing you'd expect.

I will say that I think a C version of cat2dit would be a very nice thing to have.  Maybe if I need a project to distract me after groff 1.23 is released...or maybe some enthusiastic volunteer will show up.

In fact, a C version of cat2dit that would run on V7 Unix under SIMH would, in my imagination, kick ass.  (People really wanted one, back in the '80s.)  Then there would be no data migration issue or octal dumping to mess with.  Run SIMH in a terminal window with enough scrollback and you could just copy and paste the entire ditroff output to wherever you wanted.

A tool like this is necessary if we are to hew to Ralph's rather stentorian declaration that groff must copy "historic" behavior, at least if that history is to encompass aspects of V7 troff behavior that its nroff output cannot reveal and which CSTR #54 doesn't clearly specify.

My own view is that it's worthwhile to have V7 troff's behavior exposed for analysis as easily as possible, but that groff should feel free to deviate from that behavior where cost/benefit considerations point that direction.  Plenty of "Implementation Differences" are already documented in our Texinfo manual and groff_diff(7) page; some of them I have added myself in the course of studying groff and its cousins over the past few years.

Anyway, the good news is that I have resolved my questions to my satisfaction, have a demo, and I'll post my findings to the list.

G. Branden Robinson <gbranden>
Project Administrator
Fri 19 Nov 2021 08:54:16 AM UTC, comment #11: 

To make the record here complete: bug #61437 precipitated this issue being revisited and the conclusion of comment #1 overturned.

Dave <barx>
Project Member
Tue 16 Nov 2021 05:42:05 PM UTC, comment #10: 

CSTR 54 is correct.  Branden's 'guess' is wrong and no change from Bell Labs troff's normal and documented behaviour should be made based on it.  vflag is used to indicate the motion is vertical.  It is nothing to do with forcing a unit calculation to be vees.  The code of makem(), quoted but not analysed above, makes this clear.

groff's behaviour and documentation should copy both CSTR 54 and the historic behaviour which as are in agreement are therefore correct.

Ralph Corderoy <ralph>
Tue 16 Nov 2021 07:31:30 AM UTC, comment #9: 

I don't have any analysis yet, but here's a C program to undump "od -b".

#include <stdbool.h> // bool, false, true
#include <stdio.h> // feof(), getline(), stdin, fprintf(), stderr, perror()
#include <stdlib.h> // malloc(), free(), exit(), EXIT_FAILURE, EXIT_SUCCESS
#include <string.h> // strtok()

int main(int argc, char *argv[]) {
    size_t size = 0;
    const int linelen = 72; // typical od -b output
    int linecount = 0;
    char *linebuf = NULL;
    ssize_t status = 0;
    int address = 0;

    // Be able to handle '*' lines.
    bool do_repeat = false;
    unsigned int cache[16] = { 0 };
    int last_seen_address = 0;

    linebuf = malloc(linelen);
    if (NULL == linebuf) {
        fprintf(stderr, "unodb: could not allocate memory\n");
        exit(EXIT_FAILURE);
    }

    while ((status = getline(&linebuf, &size, stdin)) > 0) {
        linecount++;
        // If the line is "*", it's special.
        if (linebuf[0] == '*') {
            if (0 == linecount) {
                (void) fprintf(stderr, "unodb: corrupt file begins with '*'\n");
                exit(EXIT_FAILURE);
            }
            last_seen_address = address;
            do_repeat = true;
            continue;
        }

        char *bufptr = NULL;
        bufptr = strtok(linebuf, " \n");

        if (!sscanf(bufptr, "%07o", &address)) {
            (void) fprintf(stderr, "unodb: error parsing address at line %d\n",
                           linecount);
        }

        // Are we resuming after a * line?
        if (do_repeat) {
            // Write the cached 16 bytes until we catch up to the new address.
            // We've already processed 16 bytes after the last seen address.
            int written_address = last_seen_address + 16;
            do {
                for (int i = 0; i < 16; i++) {
                    (void) putchar(cache[i]);
                }
                written_address += 16;
            } while (written_address < address);
            do_repeat = false;
        }

        // Read sequences of up to 16 byte values in octal.
        for (int i = 0; i < 16; i++) {
            unsigned int byte = 0;
            bufptr = strtok(NULL, " \n");
            if (NULL == bufptr) {
                break;
            }
            if (!sscanf(bufptr, "%03o", &byte)) {
                (void) fprintf(stderr, "unodb: error parsing byte %d on line"
                               " %d\n", (i + 1), linecount);
            }
            cache[i] = byte;
            (void) putchar(byte);
        }
    }

    if (!feof(stdin)) {
        perror("unodb: ");
    }

    free(linebuf);
    exit(EXIT_SUCCESS);
}

// vim:set cin et sw=4 ts=4 tw=80:

In any case anyone ever needs it.  The 1 or 2 I found on the Web were crap.  This produces round-trip sound octal dumps, so I'm fairly sure it's not total crap.

G. Branden Robinson <gbranden>
Project Administrator
Tue 16 Nov 2021 06:13:34 AM UTC, comment #8: 

> Excellent!  Except...maybe my JS environment is too old?


`Array.prototype.flat` was only added to the ECMAScript spec in 2019, so it's possible your version of Node.js is outdated. If your distro's lagging behind (i.e., upgrading doesn't help), you can add this to the top of the script to polyfill the missing method:

if("function" !== typeof [].flat)
        Array.prototype.flat = function(depth = 1){
                const {forEach} = this;
                const results = [];
                const fn = (obj, depth) =>
                        forEach.call(obj, x => depth > 0 && Array.isArray(x)
                                ? fn(x, depth - 1)
                                : results.push(x));
                fn(this, depth);
                return results;
        };

> but you might try the archives of the TUHS list.


Thanks! I'll have a poke around.

John Gardner <alhadis>
Tue 16 Nov 2021 05:32:37 AM UTC, comment #7: 

comment #6:

> > Now all I need is a way to get V7 troff CAT output out of the SIMH instance I'm running.
> > […]
> > Maybe I can od(1) it and then parse it back into a real stream. 
>
> That's exactly what I do. I use this to "import" binary files in a running SIMH session, and this to unpack the output of `od -b` (which I copy+paste from my terminal).


Excellent!  Except...maybe my JS environment is too old?

$ unod otroff.odbout
/home/branden/bin/unod:24
  .flat();
   ^

TypeError: fs.readFileSync(...).trim(...).split(...).map(...).map(...).flat is not a function
    at Object.<anonymous> (/home/branden/bin/unod:24:4)
    at Module._compile (internal/modules/cjs/loader.js:778:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:789:10)
    at Module.load (internal/modules/cjs/loader.js:653:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:593:12)
    at Function.Module._load (internal/modules/cjs/loader.js:585:3)
    at Function.Module.runMain (internal/modules/cjs/loader.js:831:12)
    at startup (internal/bootstrap/node.js:283:19)
    at bootstrapNodeJSCore (internal/bootstrap/node.js:623:3)

Any ideas?

> It feels amateurish and hacky, so I've recently been reading up on SIMH configuration and if there's a way of "mounting" a directory on the host system so that files can be shared with a running SIMH session (a la, VirtualBox). I'll let you know if I succeed...


I haven't heard of anyone doing anything like this, but you might try the archives of the TUHS list.

https://minnie.tuhs.org/mailman/listinfo/tuhs

G. Branden Robinson <gbranden>
Project Administrator
Tue 16 Nov 2021 04:55:42 AM UTC, comment #6: 

> Now all I need is a way to get V7 troff CAT output out of the SIMH instance I'm running.
> […]
> Maybe I can od(1) it and then parse it back into a real stream. 


That's exactly what I do. I use this to "import" binary files in a running SIMH session, and this to unpack the output of `od -b` (which I copy+paste from my terminal).

It feels amateurish and hacky, so I've recently been reading up on SIMH configuration and if there's a way of "mounting" a directory on the host system so that files can be shared with a running SIMH session (a la, VirtualBox). I'll let you know if I succeed...

John Gardner <alhadis>
Tue 16 Nov 2021 04:16:43 AM UTC, comment #5: 

Hi, John!

comment #4:

> > because I have nothing that can interpret the Graphic Systems C/A/T command stream the former produces.
>
> I do:
>

> $ cat2dit -Tps   < output.cat | grops
> $ cat2dit -Tpost < output.cat | dpost
>
> # Unscaled translation of C/A/T output
> $ cat2dit < output.cat

>
> Based on https://github.com/Alhadis/otroff/tree/cat-driver
>
> https://github.com/Alhadis/otroff/tree/cat-driver


Terrific!

Now all I need is a way to get V7 troff CAT output out of the SIMH instance I'm running.  No networking in those days.

Maybe I can od(1) it and then parse it back into a real stream.  Keeping in mind that there is a thing called "PDP-endianness".  Sounds tedious.

G. Branden Robinson <gbranden>
Project Administrator
Tue 16 Nov 2021 01:16:44 AM UTC, comment #4: 

> because I have nothing that can interpret the Graphic Systems C/A/T command stream the former produces.


I do:

$ cat2dit -Tps   < output.cat | grops
$ cat2dit -Tpost < output.cat | dpost

# Unscaled translation of C/A/T output
$ cat2dit < output.cat

Based on https://github.com/Alhadis/otroff/tree/cat-driver

https://github.com/Alhadis/otroff/tree/cat-driver

John Gardner <alhadis>
Sun 23 May 2021 05:53:13 AM UTC, comment #3: 

comment #1:

> Rule #1 of the Murray Hill Unix Room was: don't write comments.
>
> But if you do, remember Rule #2: make them misleading.


And I presume Rule #3 was "see Rule #3" in order to set up an infinite loop that prevented any subsequent rules from ever being reached.

Thanks for the historical deep dive.

Dave <barx>
Project Member
Fri 21 May 2021 07:32:17 AM UTC, comment #2: 

commit be0bd23b1b9170ca31dec4e151a65cf2c333fe6b
Author:     G. Branden Robinson <g.branden.robinson@gmail.com>
AuthorDate: Thu May 20 19:09:10 2021 +1000
Commit:     G. Branden Robinson <g.branden.robinson@gmail.com>
CommitDate: Thu May 20 19:09:10 2021 +1000

    doc/groff.texi (Page Motions): Add CSTR#54 errata.

    Thanks to Dave Kemper for noting these in Savannah #60602.

commit c6a01c81aec6b0fa8fececc670ea4c553e8ab651
Author:     G. Branden Robinson <g.branden.robinson@gmail.com>
AuthorDate: Thu May 20 18:47:37 2021 +1000
Commit:     G. Branden Robinson <g.branden.robinson@gmail.com>
CommitDate: Thu May 20 18:47:37 2021 +1000

    groff(7): Fix error in \d, \r, \u descriptions.

    * man/groff.7.man (Escape Sequences/Escape short reference): Fix errors
      in descriptions of \d, \r, \u escapes; they move in vees, not ems.

    Fixes <https://savannah.gnu.org/bugs/?60602>.

G. Branden Robinson <gbranden>
Project Administrator
Thu 20 May 2021 08:25:43 AM UTC, comment #1: 

Question 1: Which of the groff documentation's conflicting claims is correct?

These escapes all create new `vmotion_node` objects (src/roff/troff/input.cpp:2030,2160,2185).

The `vmotion` class's constructor is defined to take a first parameter of type `vunits` (src/roff/troff/node.h:352).

The implementation doesn't appear to work with vees per se; instead it grabs the "size" from the current environment and performs arithmetic on that.  That size is in basic units (src/roff/troff/env.h:277).

Nevertheless, what our Texinfo manual says should be strictly equivalent to what the implementation does, as I understand these matters.

So we need to fix groff(7) to say 'v' instead of 'm' in those places.

Question 2: What did AT&T do?

Both versions of CSTR #54 available to me (the October 1976 one with the 1-page May 1977 update sheet, and Kernighan's November 1992 revision) say ems, as you noted.

So what did the source code do?

Our search begins in Version 7 Unix's usr/src/cmd/troff/n1.c.

   599                  switch(k){
[...]
   651                          case 'u':       /*half em up*/
   652                          case 'r':       /*full em up*/
   653                          case 'd':       /*half em down*/
   654                                  i = sethl(k);
   655                                  break;

Look!  It says "ems" right there, so that's gotta be right.  But, just for grins, so what's this `sethl()` thing?  It has different implementations for nroff and troff.

n6.c:sethl(k)
t6.c:sethl(k)

The troff one will probably be more capable, so let's check that one first.

   430  sethl(k)
   431  int k;
   432  {
   433          register i;
   434
   435          i = 3 * (pts & 077);
   436          if(k == 'u')i = -i;
   437          else if(k == 'r')i = -2*i;
   438          vflag++;
   439          i = makem(i);
   440          vflag = 0;
   441          return(i);
   442  }

Elsewhere, in tdefs.h, we find a suggestive preprocessor directive.

tdef.h:#define EM (6*(pts&077))

This jibes with this `sethl()` function grabbing 3 ems', or half a vee's, worth of basic units for the apparent default case of 'd', flipping its sign if the inbound parameter was 'u', and doubling it and flipping its sign if that parameter was 'r' (so, one full negative vee of motion).

No comment explains what this global `vflag` means, but I'm guessing it means "interpret integer as vees when doing unit conversions".  The very next function, `makem()` indeed appears to be such a unit conversion function.

   443  makem(i)
   444  int i;
   445  {
   446          register j;
   447
   448          if((j = i) < 0)j = -j;
   449          j = (j & ~MOTV) | MOT;
   450          if(i < 0)j |= NMOT;
   451          if(vflag)j |= VMOT;
   452          return(j);
   453  }

I admit that I am not going to dig into all these bit operations.  I can surmise that `pts` always gets its upper bits masked off because octal 100 is decimal 40, and the C/A/T couldn't operate at a point size larger than 36.  Possibly some juicy information was stored in those bits and/or this was done to avoid the tedium of validating the input.

So `makem` is a function that takes basic units (I think) and coverts them to ems.  Except when it converts them to vees because of a magic global flag.  <slow nod>

I can't test V7 Unix troff per se; only its nroff, because I have nothing that can interpret the Graphic Systems C/A/T command stream the former produces.

But I'm pretty confident that you're right, that CSTR #54 was in error, and I think we can also make an informed speculation regarding how the error crept into the manual.

Rule #1 of the Murray Hill Unix Room was: don't write comments.

But if you do, remember Rule #2: make them misleading.

Thanks for the report, Dave!

G. Branden Robinson <gbranden>
Project Administrator
Thu 13 May 2021 07:45:39 PM UTC, original submission:  

The Texinfo manual and groff(7) do not agree on precisely what the three vertical-motion escapes \r, \u, and \d do.

Texinfo (doc/groff.texi) documents them as moving up 1v, up .5v, and down .5v, respectively.

groff(7) (man/groff.7.man) documents them as moving up 1m, up .5m, and down .5m, respectively.

CSTR #54 (section 11.1) defined the three in m's.  I haven't verified whether this corresponds with historical behavior.  Being vertical motions, v's make more sense.

Dave <barx>
Project Member

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by ralph (Posted a comment)
  • -email is unavailable- added by alhadis (Posted a comment)
  • -email is unavailable- added by gbranden (Posted a comment)
  • -email is unavailable- added by barx (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

     

     

     

    Follow 7 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2021-11-23 gbranden StatusFixed => Invalid
        Planned Release1.23.0 => None
    2021-05-21 gbranden StatusConfirmed => Fixed
        Open/ClosedOpen => Closed
        Planned ReleaseNone => 1.23.0
    2021-05-20 gbranden StatusNone => Confirmed
        Assigned toNone => gbranden

    Back to the top


    Powered by Savane 3.9