bugGNU roff - Bugs: bug #64043, [ms] mixing formatting requests...

 
 

bug #64043: [ms] mixing formatting requests with macro calls produces different unspecified behavior with groff than with AT&T troff

Submitter:  None
Submitted:  Thu 13 Apr 2023 07:11:15 PM UTC
   
 
Category:  Macro package ms Severity:  3 - Normal
Item Group:  Incorrect behaviour Status:  Rejected
Privacy:  Public Assigned to:  gbranden
Open/Closed:  Closed Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Sat 06 Jul 2024 03:00:02 AM UTC, comment #14: 

comment #13:

> Sounds good.  While the bug has been Rejected for over a year, the discussion seemed have been left in limbo, so I wanted to make sure everything that might have been resolved has been resolved.


Fair.
 

> Only one other potentially loose thread that I see: one of your emails includes the statement "I can see a reason for displays (and equations displayed with EQ/EN) in ms documents to permit pre-heading space to accumulate with inter-display distance.  For that matter, we could expose a new register to enable user control of pre-heading vertical space."  This appears not to have been done (s.tmac has had only four commits since that was written, none of which say anything about a new register).  Should it be?


Not without an expression of demand, IMO.  Which doesn't rule out that I'll feel the demand myself when crafting a future ms document.  :)

G. Branden Robinson <gbranden>
Group administrator
Sat 06 Jul 2024 02:44:57 AM UTC, comment #13: 

Sounds good.  While the bug has been Rejected for over a year, the discussion seemed have been left in limbo, so I wanted to make sure everything that might have been resolved has been resolved.

Only one other potentially loose thread that I see: one of your emails includes the statement "I can see a reason for displays (and equations displayed with EQ/EN) in ms documents to permit pre-heading space to accumulate with inter-display distance.  For that matter, we could expose a new register to enable user control of pre-heading vertical space."  This appears not to have been done (s.tmac has had only four commits since that was written, none of which say anything about a new register).  Should it be?

Dave <barx>
Group Member
Sat 06 Jul 2024 01:20:40 AM UTC, comment #12: 


comment #9:

> Branden, was this intended to address the above-quoted documentation update you offered to make?


Yes.

G. Branden Robinson <gbranden>
Group administrator
Sat 06 Jul 2024 01:19:12 AM UTC, comment #11: 

I wasn't too impressed with this report, which attempted to use the Groff Mission Statement as a bludgeon for the reporter to get his way.


Backward compatibility with existing documents and usage will remain a top priority, as will avoiding feature-bloat and increased  overheads.


Possibly that aspect of the statement promises or implies too much; over my years of contributing to groff I've observed and noted changes in Version 7, DWB, and Heirloom versions of ms, mm, and man (where applicable -- mm wasn't in Version 7) that all could be cited as models against which "backward compatibility" could be held as the highest "priority".  (One of those differences in ms came up in the linked discussion.)

Adding mandoc(1) to the mixture, we've observed differences between its mdoc and that of Net/2 BSD as well.

So what to do when coequally canonical sources are in conflict?

You make a decision and document things.

G. Branden Robinson <gbranden>
Group administrator
Sat 06 Jul 2024 12:50:59 AM UTC, comment #10: 

Additional discussion on the email list can be found in the threads starting at:

Christof Meerwald drove these discussions.  He has has neither confirmed nor denied that he is this bug report's submitter, but the points raised by both (and the writing styles) are largely the same.

Dave <barx>
Group Member
Fri 05 Jul 2024 09:10:24 PM UTC, comment #9: 

comment #6:

> The application of inter-display distance after equations does not
> appear to be documented in groff, however, and I'm happy to do that.


I'm not sure whether this was done.  Commit 1c39f5d80 (pushed about a month after comment #6 was posted here) included "Describ[ing] the handling of display distance in more detail."  It doesn't address equations specifically, but they may be included under the umbrella of "display macros."

Here's a recap of the relevant changes, which were made in the Texinfo manual, the ms.ms document, and the groff_ms(7) man page.

The clause "The distance stored in the 'DD' register is inserted before and after each pair of display macros, replacing any adjacent inter-paragraph distance" had the phrase after the comma removed.  A new sentence was then inserted after this one, expanding that phrase into: "In 'groff' 'ms', this distance replaces any adjacent inter-paragraph distance or subsequent spacing prior to a section heading."  (The wording has since been tweaked but retains that essence.)

Branden, was this intended to address the above-quoted documentation update you offered to make?

Submitter, does the text added in this commit address the changes in vertical space you observe between groff 1.22.4 and 1.23.0?

Dave <barx>
Group Member
Fri 14 Apr 2023 07:08:19 AM UTC, comment #8: 


So, are we talking about vertical space inside section 8, or vertical space right before section 11?


Your screenshots showed the spacing difference for section 8, my screenshots showed the spacing difference right before section 11.

In both cases I observed that the spacing in the original document was about twice the spacing you get after your changes (this can't be explained by slightly different magnification)


Again, see the attached screenshot.


and then


troff output (before post-processing) would be more informative.


let's talk about shifting goalposts...


This is the only explanation I can imagine for your utter indifference to the complete loss of text in the original document when formatted by AT&T Version 7 Unix troff.


I still fail to see what an obscure bug in AT&T Version 7 Unix troff has to do with these changes.



This register assignment is ill-formed.


ok, just use


.nr PS 12p


instead and you get ms-3.png (groff 1.22.4) vs ms-4.png (groff 1.23). The vertical space between "3" and "4" as well as between "6" and "7" is now gone with groff 1.23.


Anonymous
Fri 14 Apr 2023 12:42:29 AM UTC, comment #7: 

Oh, I forgot to mention after that long detour through Lesk's 1979 tmac.s...

Even though your display distance is 0, there is still vertical space after the equation in your small sample document because a section heading follows.

I predict, but haven't fired up SIMH to check, that Unix V7 ms would permit the space after the displayed equation to accumulate with sectioning space, which would be (IMO) incorrect as a matter of pleasant page layout.

Moreover, this directly addresses the exhibits attached as ms-1.png and ms-2.png.  I think groff is doing the right thing here, and indeed, this issue doesn't have anything to do with the changes I made consequent to bug #62688--groff 1.22.4 behaves the same way.

These are indeed some highly stochastic goal posts.

G. Branden Robinson <gbranden>
Group administrator
Fri 14 Apr 2023 12:33:32 AM UTC, comment #6: 

comment #3:

> Attached screenshots ms-1.png and ms-2.png show vertical space of 25 pixels vs 12 pixels - also seems significant and quite obvious to see.
>
>


comment #4:

> Also failing to see where that's mixing formatting requests with macro calls...


That's because you're shifting the goal posts as I said.  The original complaint "you" (anonymous) reported in comment #0 cited a mailing list message which included the following.


Looking at page 158, in "8. Braces for Grouping", I see quite a bit of
white-space before

  "Rule: Braces can always be used to force EQN to treat something as
   a unit,"

which seems to correspond to

.EQ
e sup {i omega t}
.EN
.sp
Rule:  Braces can
.ul
always
be used to force
.UC EQN
to treat something as a unit,

in the source code. So the ".sp" clearly does affect spacing here
(after .EN).


So, are we talking about vertical space inside section 8, or vertical space right before section 11?

I expect the specifics of your complaint to shift for as long as I care to entertain this discussion.


> .nr PS 1v


This register assignment is ill-formed.  Vees are not a conventional unit for expressing type size.  Points (or, in groff, scaled points) are.  Please see the section "Measurements" of the groff Texinfo manual.

At best, '.nr PS 1v' is nilpotent.  At worst, it has varying consequences on terminal vs. typesetting devices that could be difficult to predict.

Omitting it and rendering the following document with groff 1.22.4 and then 1.23.0.rc3...


> .nr DD 0v
> .LP
> 1
> .NH
> 2
> .EQ
> 3
> .EN
> .NH
> 4
> .LP
> 5
> .DS
> 6
> .DE
> .NH
> 7
> .LP
> 8


I see nothing unexpected.

Vertical ("blank") space is evident nowhere except after the "1" paragraph (prior to a section heading).

In groff, inter-display distance is applied after (displayed, as opposed to inline) equations.  This is an old decision, going back to groff 1.02 (June 1991) or earlier.  Your inter-display distance is zero, so no vertical space appears after it.

Looking at Lesk's 1979 tmac.s, I see that groff's behavior is consistent with, but more flexible than, his.  This Unix V7 ms also puts no vertical space before (displayed) equations, but does put either .5v or 1v of space after them, depending on a long chain of conditional tests that I haven't bothered to trace in detail, but appear to be checking to see if the equation was empty by various means (testing the diversion height, the output line number, vertical drawing position in a diversion, and so forth).  groff performs similar tests but indirects the vertical space amount through the package's `DD` register, which is set to--surprise!--0.5v or 1v depending on the type of output device.

The application of inter-display distance after equations does not appear to be documented in groff, however, and I'm happy to do that.  Do you agree to redraw the scope of this ticket to that problem?  Since it's only a doc change, I may even be able to get it in for 1.23.0 final.

G. Branden Robinson <gbranden>
Group administrator
Fri 14 Apr 2023 12:04:27 AM UTC, comment #5: 

Assuming for the sake of argument that the same "anonymous" is replying as filed the initial report...

comment #2:

> In your screenshots I see 13 pixels vs. 26 pixels of vertical space
>  before "Rule: Braces can always be..." - this seems significant to me (and I thought that difference was obvious enough).


The magnifications of the two windows is slightly different, and may not be a reliable guide.  troff output (before post-processing) would be more informative.

But that is still somewhat beside the point.

There is no specification of whether interdisplay distance should accumulate with other sources of vertical space, and my recollection is making the change improved the document's rendering in other respects.  (Otherwise, I wouldn't have bothered.)
 

> I fail to see the relevance of "the equation at lines 437 to 439 is absent, and the top of the next column not correctly aligned (see bug #62686)" in the context of this bug report.


Then let's return to...

original submission:

> Other examples of documents that are negatively affected by that change can be found in https://github.com/n-t-roff/Plan9_troff/tree/master/sys/doc


This remark ruled in three unrelated documents; I think content on the same column of the same page of the same document is well within scope.

But I think the objective here is to keep shifting the goal posts until I revert the change, and you will cast about for rationales until you can find one.

This is the only explanation I can imagine for your utter indifference to the complete loss of text in the original document when formatted by AT&T Version 7 Unix troff.

Rewrite your document to resort to formatter requests as little as possible, and only where the macro package exposes no locus of control for the typesetting feature in question.  This is how macro packages have always worked, and it is a lesson that *roff users have imparted to each other with increasing emphasis over the years.

Allman's me documentation rules in a limited set of formatter requests that "can be used with impunity" (importantly, even then only after an me macro call to initialize the package) and the DWB 3.3 mm manual says that "one seldom needs to use these requests directly".  groff man(7), mdoc(7), and mandoc(1) discourage the use of requests at all (the last, strongly).  Peter Schaffter's mom(7) package interposes its comprehensive interface between the formatter and the document.

People learned the hard way that they get into unportable trouble when mixing formatter requests with calls to the macros of full-service packages.

If you want to format Plan 9 documents precisely as Plan 9 does, I suggest using Plan 9's troff.

In the meantime, you may wish to respond to the people who offered feedback on the groff ms(7) changes at the time, to point out how poor their vision or judgment was.

https://lists.gnu.org/archive/html/groff/2022-07/msg00000.html

G. Branden Robinson <gbranden>
Group administrator
Thu 13 Apr 2023 09:56:12 PM UTC, comment #4: 

Also failing to see where that's mixing formatting requests with macro calls...


.nr PS 1v
.nr DD 0v
.LP
1
.NH
2
.EQ
3
.EN
.NH
4
.LP
5
.DS
6
.DE
.NH
7
.LP
8


Anonymous
Thu 13 Apr 2023 09:32:35 PM UTC, comment #3: 

Attached screenshots ms-1.png and ms-2.png show vertical space of 25 pixels vs 12 pixels - also seems significant and quite obvious to see.


Anonymous
Thu 13 Apr 2023 09:19:45 PM UTC, comment #2: 

In your screenshots I see 13 pixels vs. 26 pixels of vertical space
 before "Rule: Braces can always be..." - this seems significant to me (and I thought that difference was obvious enough).

I fail to see the relevance of "the equation at lines 437 to 439 is absent, and the top of the next column not correctly aligned (see bug #62686)" in the context of this bug report.


Anonymous
Thu 13 Apr 2023 08:50:41 PM UTC, comment #1: 


> the changes made under bug #62688 actually do the exact opposite of what is claimed in that bug


No, they don't.  The summary of the bug is "inter-display distance accumulates with other vertical space--it should not".

And the resolution performs exactly what one would expect from the summary; vertical space between displays is no longer cumulative.

The is discussion about this on the mailing list remains open with several unresolved questions, including about which formatter requests are "safe" for use in ms documents, and what users can expect when resorting to them.

Quoting the cited email message, since the submitter could not be bothered to making a concrete, measurable claim in this bug report:


Hmm, just been trying to look a bit into this.

What I found was
https://mail.gnu.org/archive/html/groff/2022-07/msg00000.html
https://mail.gnu.org/archive/html/groff/2022-07/msg00002.html and
http://www.bitsavers.org/pdf/att/unix/7th_Edition/UNIX_Programmers_Manual_Seventh_Edition_Vol_2_1983.pdf

Looking at page 158, in "8. Braces for Grouping", I see quite a bit of
white-space before

  "Rule: Braces can always be used to force EQN to treat something as
   a unit,"

which seems to correspond to

.EQ
e sup {i omega t}
.EN
.sp
Rule:  Braces can
.ul
always
be used to force
.UC EQN
to treat something as a unit,

in the source code. So the ".sp" clearly does affect spacing here
(after .EN).


"Quite a bit of white-space" (vertical space is meant) does not tell me exactly how much vertical space is present or how much should be expected.  To my eye, the amount of vertical space seems comparable; see attachments.

On the same page of the same document, AT&T troff appears to have  lost an entire equation from the source.


   432  Braces can occur within braces if necessary:
   433  .P1
   434  e sup {i pi sup {rho +1}}
   435  .P2
   436  is
   437  .EQ
   438  e sup {i pi sup {rho +1}}
   439  .EN
   440  The general rule is that anywhere you could use some single
   441  thing like


> the changes result in less faithful rendering of historic documents.


In authentically rendered Version 7 Unix typesetter output, the equation at lines 437 to 439 is absent, and the top of the next column not correctly aligned (see bug #62686).  (I suspect these are related problems.)

Again, see the attached screenshot.

I decline to resurrect this authentic misbehavior.

> Other examples of documents that are negatively affected by that change can be found in https://github.com/n-t-roff/Plan9_troff/tree/master/sys/doc


Nope.  Vaguely waving one's hands in the direction of a huge corpus of documents, and saying that they are somehow "negatively affected" with no specifics, no analysis, and no historical perspective is not how collaborative software development is undertaken.  It is simply hostile.

Closing as rejected.


G. Branden Robinson <gbranden>
Group administrator
Thu 13 Apr 2023 07:11:15 PM UTC, original submission:  

As has been shown in https://lists.gnu.org/archive/html/groff/2023-04/msg00097.html , the changes made under bug #62688 actually do the exact opposite of what is claimed in that bug - the changes result in less faithful rendering of historic documents.

Other examples of documents that are negatively affected by that change can be found in https://github.com/n-t-roff/Plan9_troff/tree/master/sys/doc

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #54624:  ms-3.png added by None (3KiB - image/png)
file #54625:  ms-4.png added by None (3KiB - image/png)
file #54622:  ms-1.png added by None (16KiB - image/png)
file #54623:  ms-2.png added by None (10KiB - image/png)
file #54619:  savannah_64043-1.png added by gbranden (298KiB - image/png)
file #54620:  savannah_64043-2.png added by gbranden (138KiB - image/png)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by barx (Posted a comment)
  • -email is unavailable- added by gbranden (Updated the item)
  • -email is unavailable- added by gbranden (mailing list discussant)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 12 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2024-07-05 barx Summarymixing formatting requests with macro calls produces different unspecified behavior with groff than with AT&amp;T troff [ms] mixing formatting requests with macro calls produces different unspecified behavior with groff than with AT&T troff
    2023-04-14 None Attached File- Added ms-3.png, #54624
        Attached File- Added ms-4.png, #54625
    2023-04-13 None Attached File- Added ms-1.png, #54622
        Attached File- Added ms-2.png, #54623
    2023-04-13 gbranden Summaryregression introduced with bug #62688: [ms] inter-display distance accumulates with other vertical space--it should not mixing formatting requests with macro calls produces different unspecified behavior with groff than with AT&T troff
    2023-04-13 gbranden Open/ClosedOpen Closed
    2023-04-13 gbranden Attached File- Added savannah_64043-1.png, #54619
        Attached File- Added savannah_64043-2.png, #54620
        StatusNone Rejected
        Assigned toNone gbranden
        Carbon-Copy- Added -email is unavailable-

    Back to the top

    Powered by Savane 3.15-e6e5.
    Corresponding source code