Yes, this bug does suffer from comment overload.  Comment #13 is the basic problem statement as I understand it now.  Previous discussion was about specific cases of what comment #13 illustrates, and yes, some of those examples are overblown and not particularly illuminating.

comment #18:

> I ported your reproducer to ms,

Your port doesn't actually reproduce the situation the other examples are demonstrating, at least not with a US-letter paper setting.  I get this output:

$ groff -a need1.ms 2> /dev/null
<beginning of page>
1
2
3
4
5
6

<beginning of page>
 -2-

But having all lines on the first page doesn't set up the situation where we can test the bug; for that, there needs to be a page break before line 6.

I was able to bring this about by changing the 9i to 9.3i.  And once that's done, the two files do in fact show the bug present in -ms:

$ sed s/9i/9.3i/ need1.ms | groff -a 2> /dev/null
<beginning of page>
1
2
3
4
5

<beginning of page>
 -2-
6

$ sed s/9i/9.3i/ need2.ms | groff -a 2> /dev/null
<beginning of page>
1
2

<beginning of page>
 -2-
3
4
5
6

This output is the same for me with 1.22.4 and 1.23.0.rc4.

I ported your reproducer to ms, and it doesn't reproduce.

$ cat EXPERIMENTS/need1.ms
.mso s.tmac
.nr FM 9i
.nf
1
.pev
.ptr
.tm .t=\n[.t]
.ne 3
.tm .ne=\n[.ne], .trunc=\n[.trunc], .trap=\n[.trap]
2
3
4
5
6
$ cat EXPERIMENTS/need2.ms
.mso s.tmac
.nr FM 9i
.nf
1
2
.pev
.ptr
.tm .t=\n[.t]
.ne 3
.tm .ne=\n[.ne], .trunc=\n[.trunc], .trap=\n[.trap]
3
4
5
6
$ ./build/test-groff -a EXPERIMENTS/need1.ms >| need1s.out 2>&1
$ ./build/test-groff -a EXPERIMENTS/need2.ms >| need2s.out 2>&1
$ $ diff -u need[12]s.out
--- need1s.out  2023-05-18 05:43:03.592759442 -0500
+++ need2s.out  2023-05-18 05:43:11.324731344 -0500
@@ -40,7 +40,7 @@
   maximum number of consecutive hyphenated lines: -1
   hyphenation space: 0u
   hyphenation margin: 0u
-Environment h:
+Environment ne:
   previous line length: 468000u
   line length: 468000u
   previous title length: 468000u
@@ -55,7 +55,7 @@
   family: 'T'
   space size: 12/36 em
   sentence space size: 12/36 em
-  previous line interrupted: no
+  previous line interrupted: yes
   fill mode: on
   adjust mode: both
   previous vertical spacing: 12000u
@@ -69,7 +69,7 @@
   temporary indentation: 0u
   have temporary indentation: no
   currently used indentation: 0u
-  target text length: 0u
+  target text length: 468000u
   previous text length: 0u
   total width: 0u
   total number of spaces: 0
@@ -77,12 +77,12 @@
   line tabs: no
   discarding: no
   spread flag set: no
-  hyphenation_flags: on, not last line, not last two chars
+  hyphenation_flags: on
   number of consecutive hyphenated lines: 0
   maximum number of consecutive hyphenated lines: -1
   hyphenation space: 0u
   hyphenation margin: 0u
-Environment fn:
+Environment nf:
   previous line length: 468000u
   line length: 468000u
   previous title length: 468000u
@@ -98,10 +98,10 @@
   space size: 12/36 em
   sentence space size: 12/36 em
   previous line interrupted: no
-  fill mode: on
+  fill mode: off
   adjust mode: both
   previous vertical spacing: 12000u
-  vertical spacing: 8000u
+  vertical spacing: 12000u
   previous post-vertical spacing: 0u
   post-vertical spacing: 0u
   previous line spacing: 1
@@ -124,7 +124,9 @@
   maximum number of consecutive hyphenated lines: -1
   hyphenation space: 0u
   hyphenation margin: 0u
-Environment ne:
+Environment 0:
+  current
+Environment k:
   previous line length: 468000u
   line length: 468000u
   previous title length: 468000u
@@ -139,7 +141,7 @@
   family: 'T'
   space size: 12/36 em
   sentence space size: 12/36 em
-  previous line interrupted: yes
+  previous line interrupted: no
   fill mode: on
   adjust mode: both
   previous vertical spacing: 12000u
@@ -153,7 +155,7 @@
   temporary indentation: 0u
   have temporary indentation: no
   currently used indentation: 0u
-  target text length: 468000u
+  target text length: 0u
   previous text length: 0u
   total width: 0u
   total number of spaces: 0
@@ -166,7 +168,7 @@
   maximum number of consecutive hyphenated lines: -1
   hyphenation space: 0u
   hyphenation margin: 0u
-Environment nf:
+Environment h:
   previous line length: 468000u
   line length: 468000u
   previous title length: 468000u
@@ -182,7 +184,7 @@
   space size: 12/36 em
   sentence space size: 12/36 em
   previous line interrupted: no
-  fill mode: off
+  fill mode: on
   adjust mode: both
   previous vertical spacing: 12000u
   vertical spacing: 12000u
@@ -203,14 +205,12 @@
   line tabs: no
   discarding: no
   spread flag set: no
-  hyphenation_flags: on
+  hyphenation_flags: on, not last line, not last two chars
   number of consecutive hyphenated lines: 0
   maximum number of consecutive hyphenated lines: -1
   hyphenation space: 0u
   hyphenation margin: 0u
-Environment 0:
-  current
-Environment k:
+Environment fn:
   previous line length: 468000u
   line length: 468000u
   previous title length: 468000u
@@ -229,7 +229,7 @@
   fill mode: on
   adjust mode: both
   previous vertical spacing: 12000u
-  vertical spacing: 12000u
+  vertical spacing: 8000u
   previous post-vertical spacing: 0u
   post-vertical spacing: 0u
   previous line spacing: 1
@@ -255,7 +255,7 @@
 pg@top 0
 pg@bottom      -648000
 pg*footer      -324000
-.t=60000
+.t=48000
 .ne=0, .trunc=0, .trap=pg@bottom
 <beginning of page>
 1

I don't have any idea which of those environmental differences are red herrings.  The environments are getting spat out in a different order, which may be introducing significant noise.

comment #15:

> All of the me(7) keep macros emit a break request before closing
> the diversion.  That means there's always an extra space down
> to the next baseline at the ends of the diversions.

I'm sure that's true as far as it goes, but the comment #13 examples were intended to show that the off-by-one behavior happens even without using any -me diversions, keeps, or other macro-defined structures -- even, for that matter, in the absence of using any -me features at all, save for the changing of the bottom margin.  (And that's only for example-shortening sake.  The same behavior is observable without that; you just need a lot more lines to get there.)  The mere loading of the macro package makes .ne behave slightly off.

As you note, this does not happen in plain roff, but as comment #13 notes, it does also happen with at least -mm, even without using any -mm features.  (I don't know any, save the setting of the bottom margin, which I looked up specifically for this example.)  I didn't try any other packages; two seemed sufficient to demonstrate that this is a problem beyond a single macro package.

The upshot (which I fear may have gotten lost in the minutiae) is that the proposed fix to .)b in comment #8 amounts to "lie to .ne about how many lines are needed."  But there are numerous other cases, as comment #13 demonstrates, where lying to .ne by telling it one fewer line is also necessary to produce the correct result.

So the first fix should be figuring out how to tell .ne the truth in general.  Only then, if the -1v is still needed in .)b due to diversion semantics, should that change be considered.  (But I bet it won't.)

I'll play with it some more, but I'm not making the general claim that .ne is broken in pure roff, only that it exhibits off-by-one behavior under -me (and under -mm, if Ulrich is to be believed, though I haven't tested this).  That is, it's perhaps not purely a -me bug or an .ne one, but something subtle in the interactions of these things.

comment #9:

> And the thread linked in comment #1

For the sake of completeness: the archive bot mangled Ulrich's example in this thread because it referred to internal mm macro pg@footer, which the bot mistook for an email address.  The code snippet as posted read:

.ch pg@footer
.BL
[any stuff here]
.LE
.wh \n[pg*foot-trap]u pg@footer
.BP

(If you're reading this note in the bug-groff archive, it'll have been remangled, so follow the link to the savannah bug report to see it unredacted.)

Aha.

All that excavation (and venting my spleen over George Helffrich) seems to have enabled me to unearth a mewling kitten of insight buried amid the macrology rubble.

This line has been bothering me.

.      ne \\n(dnu              \" make it all on one page

I thought it was because my fastidious mind objected to the casual, imprecise language in the comment.  But maybe it was something else.

Try this patch.

diff --git a/tmac/e.tmac b/tmac/e.tmac
index cbe4ae6a7..a20fa2d09 100644
--- a/tmac/e.tmac
+++ b/tmac/e.tmac
@@ -1072,7 +1072,7 @@
 .de )b                 \" *** end block
 .@)
 .if (\\n(bt=0):(\\n(.t<\\n(bt) \
-.      ne \\n(dnu              \" make it all on one page
+.      ne \\n(dnu-1v           \" reserve room for diversion
 .@k |k ns
 .ev                             \" return from display environment
 .nr ?V 0

The idea here is that we don't need to reserve room for the trailing line break--more precisely, for a vertical motion down to the next baseline--in the diversion, because we don't need to avoid a page break there.  It's perfectly fine with us if the page breaks right after the block.  Including that trailing break in our calculation reserves one more line than we need.

The obvious question is, does this f*** up single-line blocks (which normally have no reason to exist), exposing our math as fallacious?

No, it does not.  See attached keep3.me.  And, for the more practical 2-line case, keep4.me.

I think this constitutes an RCA and a fix.  Please let me know if you concur, or don't.

(file #52549, file #52550)

comment #5:

> comment #4:
> > I therefore do not think there is a bug here; if only half a
> > line of space is available, a full-height line cannot be typeset.
>
> Yes, but we know that a full-height line can be typeset, because when that line appears below the .(b block, a full-height line is typeset.  Moving it from below to above the block shouldn't change how many full-height lines fit on the page.

That certainly makes intuitive sense.

I went back and re-instrumented e.tmac and keep.me a bit differently, and parameterized the latter so that it was easy to control the output of lines before and after the block.  I added Boolean flags for them, so that gives us four combinations.  All I can find is internal consistency.

$ tg -me -rA1=0 -rA2=0 EXPERIMENTS/keep.me >| keep-00.ps
GBR: height of 7 lines: 84000u
GBR: before (b: nl=624000u
GBR: )b: dn=84000u, .t=90000u
GBR: height of 7-line diversion: 84000u
$ tg -me -rA1=0 -rA2=1 EXPERIMENTS/keep.me >| keep-01.ps
GBR: height of 7 lines: 84000u
GBR: before (b: nl=624000u
GBR: )b: dn=84000u, .t=90000u
GBR: height of 7-line diversion: 84000u
$ tg -me -rA1=1 -rA2=0 EXPERIMENTS/keep.me >| keep-10.ps
GBR: height of 7 lines: 84000u
GBR: before (b: nl=636000u
GBR: )b: dn=84000u, .t=78000u
GBR: height of 7-line diversion: 84000u
$ tg -me -rA1=1 -rA2=1 EXPERIMENTS/keep.me >| keep-11.ps
GBR: height of 7 lines: 84000u
GBR: before (b: nl=636000u
GBR: )b: dn=84000u, .t=78000u
GBR: height of 7-line diversion: 84000u

I observe that (1) the diversion is always the height we expect (I think this rules out confounding effects of vertical spacing or the presence/absence of an additional break); (2) adding a line before it increases the page position by 1v; and (3) that increase makes me(7) decide there is not enough room for the diversion on the page, so the break happens when and where you expect based on these numbers.

I'll attach my new keep.me (I might have to name it keep2.me).  Here's the diff to e.tmac.

diff --git a/tmac/e.tmac b/tmac/e.tmac
index cbe4ae6a7..ba8ac212f 100644
--- a/tmac/e.tmac
+++ b/tmac/e.tmac
@@ -1071,6 +1071,7 @@
 .
 .de )b                 \" *** end block
 .@)
+.tm GBR: )b: dn=\\n(dnu, .t=\\n(.tu
 .if (\\n(bt=0):(\\n(.t<\\n(bt) \
 .      ne \\n(dnu              \" make it all on one page
 .@k |k ns

> However, this may be less a -me bug and more a groff one: replacing the ".(b" and ".)b" bracketing the block with a ".ne 7" before the block exhibits the same behavior.

I'm not sure precisely what behavior we're observing.  Motivated by your remark above, I presumed that I could get groff to produce two pages from the following input.

$ cat EXPERIMENTS/ne.roff
.nf
.ne 66v
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66\c

...but I can't.  (Note the absence of a break on the very last line of the page--it makes all the difference.)

Starting from there, can you elucidate where the surprise creeps in?

(I tried to reason directly from that example, particularly including the `\c` factor, but I can't--me(7) always inserts `br` requests around diversions anyway (similarly to groff's "box" diversions), and anyway if there were a missing line break at the end, the diversion's height would be shorter.  Trying it out, adding `\c` after "fifty-two" made no difference.)

Hi Dave,

comment #3:

>   The symptom is caused by a higher footer in troff-mode than in the
> nroff-mode.
>
>   The footer in troff-mode is half a line bigger (6.5 v).
>
>   The documentation ("doc/meref.ps") mentions "6v ...[nominal]" for
> the register "bm".
>
>   It is common to have a different page layout in troff from that in
> nroff as the resolution is very different.
>
>   troff is a typesetting "machine"; nroff a typewriter, line printer
> "machine".

I believe Bjarni [quoted] is correct here.

I instrumented e.tmac's `)b` macro to report the .t (distance to next trap), dn (diversion height), and bt (me's "break threshold"), and to dump all active traps.

I also instrumented your reproducer (which I called "keep.me") to also report the distance to the next trap and the vertical line height (.v) in interesting places.

I noticed that the distance to the next trap was exactly half a line height in your original reproducer (before the slight modification that makes nroff and troff diverge).

I then went ahead with your procedure of adding one line, but modified it by typesetting the new line in 5-point type on 6-point vertical spacing.

Lo and behold, the example started working as you expected again.

I therefore do not think there is a bug here; if only half a line of space is available, a full-height line cannot be typeset.

Do you concur?

I am including my "keep.me" and the patch I applied to e.tmac to instrument it.  (It might apply with an offset--I have other experiments in the pipeline.)

$ cat EXPERIMENTS/keep.me
.nr bs 0         \" no extra space before or after blocks
.nf
.\".ig
.ps 5z
.vs 6p
Hello, world!
.vs
.ps
.\"..
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
.tm GBR: keep.me: before (b: .t=\n(.t, .v=\n(.v
.(b
forty-six
forty-seven
forty-eight
forty-nine
fifty
fifty-one
fifty-two
.tm GBR: keep.me: before )b: .t=\n(.t, .v=\n(.v
.)b
.tm GBR: keep.me: after )b: .t=\n(.t, .v=\n(.v
53
54
55
56
57
58
59
60

$ ./build/test-groff -b -ww -a -me EXPERIMENTS/keep.me | grep -FC8 page
GBR: keep.me: before (b: .t=84000, .v=12000
GBR: keep.me: before )b: .t=2147483646, .v=12000
GBR: )b: .t=84000, dn=84000, bt=0
@h      0
@f      -78000
GBR: keep.me: after )b: .t=630000, .v=12000
<beginning of page>
Hello, world!
1
2
3
4
5
6
7
--
45
 forty-six
 forty-seven
 forty-eight
 forty-nine
 fifty
 fifty-one
 fifty-two
<beginning of page>
53
54
55
56
57
58
59
60

diff --git a/tmac/e.tmac b/tmac/e.tmac
index 4f74a1966..912d43401 100644
--- a/tmac/e.tmac
+++ b/tmac/e.tmac
@@ -1071,6 +1071,8 @@
 .de )b                 \" *** end block
 .br
 .@)
+.tm GBR: )b: .t=\\n(.t, dn=\\n(dn, bt=\\n(bt
+.ptr
 .if (\\n(bt=0):(\\n(.t<\\n(bt) \
 .      ne \\n(dnu              \" make it all on one page
 .@k |k ns

Additionally, the thread mentioned in comment #1 continued for a bit off-list when Deri emailed me some PDF attachments he couldn't easily send to the list.  And thanks to his work, I've been able to home in a little more on this bug.

Because as it turns out, while the bug was in groff 1.21, and is in current groff, it has not been in every version of groff released in between.  I'm not sure which all official releases it appears in -- it is absent in at least 1.22.2 and 1.22.3 -- but I've iterated through the various versions of tmac/e.tmac / tmac/e.tmac-u (the file was renamed at some point) to find out exactly where it came and went.

The bug disappeared in commit b6906269 of this file and reappeared in commit 98b028d6.  But the interesting thing about this is that these commits also respectively introduced and then removed another bug.

As documented, all "displays" in -me (of which ".(b" is one) should have space corresponding to the \n(bs register above and below; the test case at hand explicitly sets this register to 0, specifying no extra space above or below the block.  But starting with commit b6906269, this test case began emitting an extra line above the .(b block; commit 98b028d6 fixed this.

So somehow this extra-space bug removed, or at least masked, the page-break bug as a side effect.

This bug has existed at least as far back as groff version 1.21, and continues to exist in a groff built from recent git code (GNU groff version 1.22.4.74-b400-dirty).  It was originally reported in http://lists.gnu.org/archive/html/bug-groff/2012-12/msg00011.html.  The code below assumes a paper size of "letter" (groff's default in the file /usr/share/groff/current/font/devps/DESC).

In the -me reference manual, the description for .(b says:

   If the block will not fit on the current page a new page
   is begun, unless that would leave more than \n(bt [0]
   white space at the bottom of the text.  If \n(bt is
   zero, the threshold feature is turned off.

This is not what happens in the edge case, however.  Consider:

------------------------------
.mso me.tmac     \" -me macros
.nr bs 0         \" no extra space before or after blocks
.nf
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
.(b
forty-six
forty-seven
forty-eight
forty-nine
fifty
fifty-one
fifty-two
.)b
53
54
55
56
57
58
59
60
------------------------------

With either -Tps or -Tascii, on the first page this outputs lines 1 to 45, then the seven-line block, then one more line, 53, before the page break.  This tells us that the block would fit exactly at the bottom of the page, if there were one more line above it (which would take the place of line 53 below).

So add one line somewhere above the .(b.

Now, -Tascii output does what's expected, putting line "fifty-two" at the bottom of the first page, and 53 at the top of the second.  But -Tps output puts the entire block on the second page, even though there is room for it on the first page.

An easy way to tell where the page break falls in -Tps output without having to generate and display a PostScript file is (using GNU's grep):

groff -Tps -a [test-file] | fgrep -C3 page