bugGNU roff - Bugs: bug #58736, [me] footnote breaks two-column...

 
 

bug #58736: [me] footnote breaks two-column output

Submitter:  Dave <barx>
Submitted:  Wed 08 Jul 2020 05:40:28 PM UTC
   
 
Category:  Macro package me Severity:  3 - Normal
Item Group:  Incorrect behaviour Status:  Fixed
Privacy:  Public Assigned to:  gbranden
Open/Closed:  Closed Planned Release:  1.23.0
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Sat 15 Oct 2022 01:33:58 PM UTC, comment #41: 


comment #40:

> comment #39:
> > This does seem to be working now, but in testing I've stumbled
> > into other -me footnote issues
>
> Heh, and by "other" issues, I mean rediscovering the issue as presented in this bug's original submission.  The new bug #63176 hopefully distills this down to its essence.


Yes--as you said there, the log of this bug is an eye-glazing experience.  May #58736 rest in peace!

G. Branden Robinson <gbranden>
Group administrator
Sat 15 Oct 2022 05:48:48 AM UTC, comment #40: 

comment #39:

> This does seem to be working now, but in testing I've stumbled
> into other -me footnote issues


Heh, and by "other" issues, I mean rediscovering the issue as presented in this bug's original submission.  The new bug #63176 hopefully distills this down to its essence.

Dave <barx>
Group Member
Wed 05 Oct 2022 10:47:47 PM UTC, comment #39: 

Thank you!  This does seem to be working now, but in testing I've stumbled into other -me footnote issues (besides the two you just opened bugs for), so I'll file those separately.

Dave <barx>
Group Member
Mon 03 Oct 2022 05:45:49 AM UTC, comment #38: 

I went ahead with my push.  I know Dave has no fear of re-opening a ticket if I don't get everything squared away.  ;-)


commit 4eeba76c6c2aaf1900b1047f017f03257935ad62
Author: G. Branden Robinson <g.branden.robinson@gmail.com>
Date:   Sun Sep 25 22:46:54 2022 -0500

    [me]: Fix Savannah #58736.

    * tmac/e.tmac: Initialize new register `_f`, the value of `$m` (column
      count) of the previous pending footnote, to zero, meaning there is no
      previous pending footnote.

      (@o): Emit diagnostic when flushing footnotes into a different column
      layout then the pending footnote last added.  Set `_f` to zero.

      ((f): Emit diagnostic when enqueueing footnote using a different
      column layout than the previous pending footnote used.  Set `_f` to
      `$m`.

    Fixes <https://savannah.gnu.org/bugs/?58736>.  Thanks to Dave Kemper for
    the report and discussion.


G. Branden Robinson <gbranden>
Group administrator
Sun 02 Oct 2022 08:14:27 PM UTC, comment #37: 

Thanks for the quick update!  (Maybe a little TOO quick... I was already doing the manual edits and not monitoring email when you posted this.)

I ended up adding the one new line (initializing _f) in the problem block and am running some tests now.  So, reviewing your other pending changes, it appears all I'm missing are the .do's on @err (shouldn't matter, as I never use compatibility mode) and the initialization of ?f, which I presume to be a documentary matter and not a functional one.

Dave <barx>
Group Member
Sun 02 Oct 2022 07:13:45 PM UTC, comment #36: 


comment #35:

>
> comment #34:
> > First stumbling block: your 58736.diff appears to not be against git HEAD but against your locally modified version of the tree:
> >


I should also have omitted the ChangeLog from the diff; the chances of anybody ever being able to apply a patch from my working tree to that file are slim indeed.

G. Branden Robinson <gbranden>
Group administrator
Sun 02 Oct 2022 07:11:44 PM UTC, comment #35: 


comment #34:

> First stumbling block: your 58736.diff appears to not be against git HEAD but against your locally modified version of the tree:
>


> $ patch -p1 < 58736.diff
> patching file ChangeLog
> Hunk #1 FAILED at 1.
> 1 out of 1 hunk FAILED -- saving rejects to file ChangeLog.rej
> patching file tmac/e.tmac
> Hunk #3 FAILED at 2159.
> 1 out of 3 hunks FAILED -- saving rejects to file tmac/e.tmac.rej


>
> I'll see if I can apply the changes manually, assuming nothing in this patch is dependent on uncommitted changes to e.tmac.


Urp, sorry about that.  Here's the history of that file since I last pushed to master.


$ git log --patch --reverse origin..master tmac/e.tmac
commit d8bf04bebeca3546dc6d75e0c0cf325843b82e21
Author: G. Branden Robinson <g.branden.robinson@gmail.com>
Date:   Sun Sep 25 22:34:10 2022 -0500

    [me]: Fix code style nit.

    * tmac/e.tmac: Explicitly initialize and annotate internal register
      `?f`.

diff --git a/tmac/e.tmac b/tmac/e.tmac
index e215b111c..a3b529c3e 100644
--- a/tmac/e.tmac
+++ b/tmac/e.tmac
@@ -2148,6 +2148,7 @@
 .@R ?b\"               \" pending floating keep at page bottom?
 .@R ?C\"               \" at chapter header?
 .@R ?e\"               \" in equation?
+.@R ?f\"               \" inside footnote?
 .@R ?H\"               \" suppress headers and footers next page?
 .@R ?I\"               \" has the header trap been sprung?
 .@R ?N\"               \" numbering with shorter line length?

commit 4eeba76c6c2aaf1900b1047f017f03257935ad62
Author: G. Branden Robinson <g.branden.robinson@gmail.com>
Date:   Sun Sep 25 22:46:54 2022 -0500

    [me]: Fix Savannah #58736.

    * tmac/e.tmac: Initialize new register `_f`, the value of `$m` (column
      count) of the previous pending footnote, to zero, meaning there is no
      previous pending footnote.

      (@o): Emit diagnostic when flushing footnotes into a different column
      layout then the pending footnote last added.  Set `_f` to zero.

      ((f): Emit diagnostic when enqueueing footnote using a different
      column layout than the previous pending footnote used.  Set `_f` to
      `$m`.

    Fixes <https://savannah.gnu.org/bugs/?58736>.  Thanks to Dave Kemper for
    the report and discussion.

diff --git a/tmac/e.tmac b/tmac/e.tmac
index a3b529c3e..9a222ec54 100644
--- a/tmac/e.tmac
+++ b/tmac/e.tmac
@@ -411,7 +411,12 @@
 .wh -\\n(_Bu @r
 .if \\n(?n \
 .      nm \\n(ln
+.if \\n(_f \
+.      if !\\n(_f=\\n($m \
+.              @err writing \\n($m-column footnote in \\n(_f-column \
+layout
 .|f
+.nr _f 0
 .if \\n(?n \
 .      nm
 .fi
@@ -1600,6 +1605,11 @@
 .
 .de (f                 \" *** begin footnote
 .if "\\n(.z"|f" .do @err footnotes cannot be nested
+.if \\n(_f \
+.      if !\\n(_f=\\n($m \
+.              @err queueing \\n($m-column footnote after \
+\\n(_f-column footnote
+.nr _f \\n($m
 .ie "\\n(.z"" \
 \{\
 .      nr _D \\n(dn
@@ -2149,6 +2159,7 @@
 .@R ?C\"               \" at chapter header?
 .@R ?e\"               \" in equation?
 .@R ?f\"               \" inside footnote?
+.@R _f\"               \" column count of previous footnote
 .@R ?H\"               \" suppress headers and footers next page?
 .@R ?I\"               \" has the header trap been sprung?
 .@R ?N\"               \" numbering with shorter line length?

commit af34450ffc7cf157627e81ffed043e412bd18620
Author: G. Branden Robinson <g.branden.robinson@gmail.com>
Date:   Wed Sep 28 02:33:40 2022 -0500

    [me]: Consistently call `@err` macro with `do`.

    * tmac/e.tmac (@o, n2, sk (f, ++): Do it.

    This prevents failure to call the diagnostic message macro when running
    in compatibility mode.

diff --git a/tmac/e.tmac b/tmac/e.tmac
index 9a222ec54..f00711577 100644
--- a/tmac/e.tmac
+++ b/tmac/e.tmac
@@ -413,8 +413,8 @@
 .      nm \\n(ln
 .if \\n(_f \
 .      if !\\n(_f=\\n($m \
-.              @err writing \\n($m-column footnote in \\n(_f-column \
-layout
+.              do @err writing \\n($m-column footnote in \
+\\n(_f-column layout
 .|f
 .nr _f 0
 .if \\n(?n \
@@ -642,7 +642,7 @@ layout
 .              rr |l
 .\}
 .      el \
-.              @err invalid numeric argument to 'n2': '\\$1'
+.              do @err invalid numeric argument to 'n2': '\\$1'
 .\}
 .el \
 \{\
@@ -696,7 +696,7 @@ layout
 .
 .de sk                 \" *** leave a blank page (next page)
 .if \\n(.$>0 \
-.      @err ignoring argument(s) to 'sk'
+.      do @err ignoring argument(s) to 'sk'
 .nr ?s 1
 ..
 .
@@ -1607,7 +1607,7 @@ layout
 .if "\\n(.z"|f" .do @err footnotes cannot be nested
 .if \\n(_f \
 .      if !\\n(_f=\\n($m \
-.              @err queueing \\n($m-column footnote after \
+.              do @err queueing \\n($m-column footnote after \
 \\n(_f-column footnote
 .nr _f \\n($m
 .ie "\\n(.z"" \
@@ -1920,7 +1920,7 @@ layout
 .if "\\$1"AB" \
 .      nr _0 5                 \" abstract
 .if \\n(_0=0 \
-.      @err invalid segment type to '++': '\\$1'
+.      do @err invalid segment type to '++': '\\$1'
 .nr ?R 0
 .if \\n(_0>10 \
 \{\


G. Branden Robinson <gbranden>
Group administrator
Sun 02 Oct 2022 07:08:09 PM UTC, comment #34: 

First stumbling block: your 58736.diff appears to not be against git HEAD but against your locally modified version of the tree:


$ patch -p1 < 58736.diff
patching file ChangeLog
Hunk #1 FAILED at 1.
1 out of 1 hunk FAILED -- saving rejects to file ChangeLog.rej
patching file tmac/e.tmac
Hunk #3 FAILED at 2159.
1 out of 3 hunks FAILED -- saving rejects to file tmac/e.tmac.rej


I'll see if I can apply the changes manually, assuming nothing in this patch is dependent on uncommitted changes to e.tmac.

Dave <barx>
Group Member
Mon 26 Sep 2022 03:57:48 PM UTC, comment #33: 

Thanks for your work on this!!  I'll test the patch later this week.  Regarding the concepts behind it:

comment #30:

> How about this added wrinkle?
>
> * If the footnote being added to the pending ones uses a
> different number of columns than the previous, throw a
> warning then, too.


Yes, good catch, that's another situation where output will need user fiddling.

> In other news, me(7) doesn't have warnings.
>
> Or, maybe we really should treat this as an error.


Agreed: I'm still of the mind that -me shouldn't be too reengineered to fix an edge case, so working within its existing framework as much as possible is good.  Additionally, since this is something that the user almost certainly will need to fix, an error is arguably more appropriate.  (It shouldn't be a fatal one, but it appears -me doesn't really have those either, except for one case where an @err call is followed by an .ab.)

Dave <barx>
Group Member
Mon 26 Sep 2022 03:53:08 AM UTC, comment #32: 

Hi Dave,

Here's a patch to try out.


diff --git a/tmac/e.tmac b/tmac/e.tmac
index a3b529c3e..9a222ec54 100644
--- a/tmac/e.tmac
+++ b/tmac/e.tmac
@@ -411,7 +411,12 @@
 .wh -\\n(_Bu @r
 .if \\n(?n \
 .      nm \\n(ln
+.if \\n(_f \
+.      if !\\n(_f=\\n($m \
+.              @err writing \\n($m-column footnote in \\n(_f-column \
+layout
 .|f
+.nr _f 0
 .if \\n(?n \
 .      nm
 .fi
@@ -1600,6 +1605,11 @@
 .
 .de (f                 \" *** begin footnote
 .if "\\n(.z"|f" .do @err footnotes cannot be nested
+.if \\n(_f \
+.      if !\\n(_f=\\n($m \
+.              @err queueing \\n($m-column footnote after \
+\\n(_f-column footnote
+.nr _f \\n($m
 .ie "\\n(.z"" \
 \{\
 .      nr _D \\n(dn
@@ -2149,6 +2159,7 @@
 .@R ?C\"               \" at chapter header?
 .@R ?e\"               \" in equation?
 .@R ?f\"               \" inside footnote?
+.@R _f\"               \" column count of previous footnote
 .@R ?H\"               \" suppress headers and footers next page?
 .@R ?I\"               \" has the header trap been sprung?
 .@R ?N\"               \" numbering with shorter line length?


I'm attaching it as well, along with my test cases.

(file #53755, file #53756, file #53757)

G. Branden Robinson <gbranden>
Group administrator
Mon 26 Sep 2022 03:11:32 AM UTC, comment #31: 

Hah, I can't use `?f`: Allman already does, undocumentedly (even more than normal--it's not in the big list of registers we initialize to zero with @R).

I'll find some other name.  The contours of my proposal have not changed.

G. Branden Robinson <gbranden>
Group administrator
Mon 26 Sep 2022 03:05:19 AM UTC, comment #30: 

Hi Dave,

comment #29:

> Here's my idea:
> * On changes to the number of columns (calls to .1c or .2c), do nothing extra.
> * On reaching an end-of-page trap when footnotes are pending, do the following:
> ** If the pending footnotes were all generated under the column width currently in effect, output them normally.
> ** If any pending footnotes were diverted using a different column width, throw a warning, and output them anyway.  They'll probably look ugly, but the warning lets the user know some kind of manual intervention is required to handle the situation, and given the difficulties outlined in comment #28, -me can't guess what will look best anyway.


This all makes sense to me.

Some Allman implementation notes:

All footnote storage is done with the diversion-append `da` request to the `|f` diversion.  The only thing that flushes the pending footnotes (calls the `|f` diversion) is a macro called `@o`.  It then removes `|f`.

How about this added wrinkle?

  • If the footnote being added to the pending ones uses a different number of columns than the previous, throw a warning then, too.


I believe I can keep the implementation simple by:

1. Adding a new internal register `?f` that is zero if no footnotes are pending.
2. Starting the first footnote (since the last flush) sets `?f` to `$m`, the column count.
3. Subsequent footnote appends check if `?f` equals `$m` before updating it.  If not, throw a warning. (If `?f` is zero, we're not really here; see step 2.)
4. When flushing the footnotes, perform the same check again.  If `$f` is not equal to `$m`, throw a different warning (the one you originally proposed).

In other news, me(7) doesn't have warnings.  I'll have to add them, moving most of the current `@err` into a `@diag` and distinguishing the diagnostic messages with "error:" and "warning:".

Or, maybe we really should treat this as an error.  My laziness leans toward the latter.  The users can ask for differentiated diagnostic levels if they want them.  Maybe me(7) users prefer the macro package lean and mean as it is.

What do you think?  I think I will have a patch ready soon, and may supply it to this ticket for your evaluation before you can reply, depending on how my own testing goes.

Regards,
Branden

G. Branden Robinson <gbranden>
Group administrator
Sat 13 Aug 2022 11:12:55 AM UTC, comment #29: 

Given the preceding, it seems safe to say that what to do is situational enough that -me shouldn't try to implement a general solution.  I'd be happy if it handled some simple cases better than it does, and threw warnings on more complex ones.

However, "simple to describe" may not equate to "simple to implement," so let me know if this idea is reasonable.  I don't think the implementation should be complex, because it's an edge case that apparently no one else has stumbled across in -me's 40-year history, so a lot of hard-to-maintain code to handle it doesn't seem appropriate.

Here's my idea:

  • On changes to the number of columns (calls to .1c or .2c), do nothing extra.
  • On reaching an end-of-page trap when footnotes are pending, do the following:
    • If the pending footnotes were all generated under the column width currently in effect, output them normally.
    • If any pending footnotes were diverted using a different column width, throw a warning, and output them anyway.  They'll probably look ugly, but the warning lets the user know some kind of manual intervention is required to handle the situation, and given the difficulties outlined in comment #28, -me can't guess what will look best anyway.


Forcing a page break (as proposed in, e.g., comment #22) is unlikely to work well in a continuous-rendering mode implemented by making the "page length" very long, so this proposal avoids the macro package doing that automatically.  Upon seeing warnings, the user can choose to do that herself if she feels it's the best way to handle the layout.

This proposal will require some way to keep track of the column width in effect when each footnote is diverted, which may be simple or challenging to implement.  If the latter, as noted above, I'll go back to the drawing board.  (It may be sufficient to track merely the number of columns in effect, but as .2c lets you specify the gutter width, which affects the column width, I'm assuming (though haven't tested) that simply the number matching is insufficient.  However, that's even more of an edge case, and one I don't care about at all, so if the number is easier to track, by all means do that.)

Dave <barx>
Group Member
Sun 03 Jul 2022 08:43:15 AM UTC, comment #28: 

comment #22:

> It is designed to output footnotes in multi-column mode at the
> bottom of the column, not the page.
>
> But what to do when the document returns to single-column mode
> with a footnote pending?


The general question is more complicated than that, as you can go from m-column output to n-column output without passing through single-column output on the way.  That is, the below works as expected.  (Perhaps "as expected" is too strong, since the manual is silent about whether it's supported--maybe "as hoped"?)

.2c "" 3
Column 1.
.bc
Column 2.
.bc
Column 3.
.2c "" 4
Column 1.
.bc
Column 2.
.bc
Column 3.
.bc
Column 4.

So it's more generally a question of what happens when the number of columns changes when a footnote is pending.

And I'm not sure there's a good general answer, since what the user might be expected to want is fairly situational.  If it's some kind of magazine or journal layout, where one multicolumn article ends, a full-width title introduces the next article, then a new multicolumn article begins, emitting the "footnotes" midpage upon encountering the .1c makes a lot of sense.  But if the temporary multicolumn passage is for some effect within a continuous text--maybe showing an original writing and a translation thereof side by side, for example--footnotes in the middle of the page might look odd.

The answer is also likely to be different if you're talking about terminal output rather than typeset output, especially if the terminal output is not "paginated" in any meaningful sense.

And as an implementation matter, I presume footnotes are stored using roff diversions, which store formatted text, which means they can only be stored using the current column width, not whatever unknown future column width will be in play at the bottom of the page.

So yeah, I agree with your statement:

> I think the problem arises from a gap in the macro package's design.


...and I'm beginning to see why this gap may have been left.

On the one hand, it might be nice to give the user some control over which of the various schemes we've identified (and maybe some we haven't yet) is used.  On the other, that seems like a nontrivial change to the user interface of a little-used macro package, to handle an issue that seems to have come up hardly at all in the 40-odd years the package has been around.

I sympathize with your reaching for the simplest solution--force a page break whenever the number of columns changes while footnotes are pending--but this is particularly ill suited for terminal output, where the "page length" may have been set to thousands of lines.

I'll have to give this more thought.

Dave <barx>
Group Member
Sun 06 Feb 2022 06:54:05 PM UTC, comment #27: 

Hi Dave,

Returning to this issue...

comment #23:

> comment #22:
> > I am soliciting opinions as to what `1c` should mean when
> > footnotes are pending
>
> I think there's a distinction to be made between cases where a
> footnote is pending that comes from within the .2c block vs
> that precedes the .2c block.


Agreed.

> I agree the former case introduces some ambiguity, but the
> latter is what prompted this bug report, and getting that to
> work might be a worthy starting point unless the problems are
> intractably intertwined.
>
> Typically multi-column output will be in effect for the whole
> page, so it's not an issue.  The original example in this bug
> report uses two columns temporarily for a special effect, but
> (a) the overall page is one column, and (b) the footnote is
> initialized outside the two-column region.
>
> (In a layout like this, it's really up to the user to ensure
> that the .2c block doesn't span a page break: if one occurs,
> -me will already make the output probably not what he wants.)


It sounds like the way forward is to have .2c check to see if
there are pending footnotes, and deal with them if so, as
proposed in comment #22.

Does it sound like we're on the same page?

Regards,
Branden

G. Branden Robinson <gbranden>
Group administrator
Fri 24 Dec 2021 03:38:54 PM UTC, comment #26: 


comment #25:

> comment #24:
> > commit 0f3c777f5dbfc5dc1f4668cfd8b8fd35bb818623
> > Author: G. Branden Robinson <g.branden.robinson@gmail.com>
> > Date:   Thu Dec 16 19:00:55 2021 +1100
> >
> >     tmac/tmac.am: Mark a test as XFAIL.
>
> Actually commit ca654b167ea9518366ba00fc40e5359a8ddb90c5.  No such commit as the ID cited above.
>
> > commit 21e6881d585902f711464c413681c7c56ac35bd1
> > Author: G. Branden Robinson <g.branden.robinson@gmail.com>
> > Date:   Thu Dec 16 18:54:01 2021 +1100
> >
> >     Revert "[me]: Fix Savannah #58736."
>
> Actually commit 757ea37560caf6deac85ac936496e918b3166c3d.  Branden is clearly reporting from a parallel universe.


Whoops.  Sorry, I must have done this from a working tree before I pushed.  I normally don't update Savannah tickets with resolver messages until I have pushed a batch of commits.  I guess my workflow got thrown off because I wasn't updating with a "resolver" this time.

These bogus IDs are the result of the nonstop orgy of rebases that my working tree sees, and of which precious little evidence escapes.  Until recently, it seems.

I'll try to be more careful.

G. Branden Robinson <gbranden>
Group administrator
Thu 23 Dec 2021 08:23:35 AM UTC, comment #25: 

comment #24:

> commit 0f3c777f5dbfc5dc1f4668cfd8b8fd35bb818623
> Author: G. Branden Robinson <g.branden.robinson@gmail.com>
> Date:   Thu Dec 16 19:00:55 2021 +1100
>
>     tmac/tmac.am: Mark a test as XFAIL.


Actually commit ca654b167ea9518366ba00fc40e5359a8ddb90c5.  No such commit as the ID cited above.

> commit 21e6881d585902f711464c413681c7c56ac35bd1
> Author: G. Branden Robinson <g.branden.robinson@gmail.com>
> Date:   Thu Dec 16 18:54:01 2021 +1100
>
>     Revert "[me]: Fix Savannah #58736."


Actually commit 757ea37560caf6deac85ac936496e918b3166c3d.  Branden is clearly reporting from a parallel universe.

Dave <barx>
Group Member
Thu 16 Dec 2021 12:16:38 PM UTC, comment #24: 


commit 0f3c777f5dbfc5dc1f4668cfd8b8fd35bb818623
Author: G. Branden Robinson <g.branden.robinson@gmail.com>
Date:   Thu Dec 16 19:00:55 2021 +1100

    tmac/tmac.am: Mark a test as XFAIL.

    * tmac/tmac.am (tmac_XFAIL_TESTS): Mark test
      "e_footnotes-work-with-columns.sh" as expected to fail (as it does
      now, consequent to a reverted commit).

    See discussion in Savannah #58736.

commit 21e6881d585902f711464c413681c7c56ac35bd1
Author: G. Branden Robinson <g.branden.robinson@gmail.com>
Date:   Thu Dec 16 18:54:01 2021 +1100

    Revert "[me]: Fix Savannah #58736."

    This reverts commit ab55f50af0599028bf7d3fd0fc50358471595b3c.

    It had undesired side effects.  The interactions of traps, footnotes,
    multi-column mode, and extreme page lengths are complex (and were not
    all completely settled even in 4.4BSD me, if the output of Heirloom
    Doctools troff is any indication).

    tmac/tests/e_footnotes-work-with-columns.sh fails at this commit.


comment #10:

> commit ab55f50af0599028bf7d3fd0fc50358471595b3c
> Author: G. Branden Robinson <g.branden.robinson@gmail.com>
> Date:   Fri Aug 13 09:32:31 2021 +1000
>
>     [me]: Fix Savannah #58736.
>
>     * tmac/e.tmac (1c): Fix problem when combining multiple columns,
>       footnotes, and return to single-column mode: do not place the
>       single-column text below the footnote.  Space to the end of the page
>       (\n[nl]), not the page "high-water mark" (\n[.h]).  Thanks to Bjarni
>       Ingi Gislason for the suggestion.
>
>     Fixes <https://savannah.gnu.org/bugs/?58736>.


G. Branden Robinson <gbranden>
Group administrator
Wed 15 Dec 2021 04:19:22 AM UTC, comment #23: 

comment #22:

> I am soliciting opinions as to what `1c` should mean when
> footnotes are pending


I think there's a distinction to be made between cases where a footnote is pending that comes from within the .2c block vs that precedes the .2c block.  I agree the former case introduces some ambiguity, but the latter is what prompted this bug report, and getting that to work might be a worthy starting point unless the problems are intractably intertwined.

Typically multi-column output will be in effect for the whole page, so it's not an issue.  The original example in this bug report uses two columns temporarily for a special effect, but (a) the overall page is one column, and (b) the footnote is initialized outside the two-column region.  (In a layout like this, it's really up to the user to ensure that the .2c block doesn't span a page break: if one occurs, -me will already make the output probably not what he wants.)

Dave <barx>
Group Member
Wed 15 Dec 2021 12:29:35 AM UTC, comment #22: 

Returning to the original report now having learned a little about how me(7) works, I think the problem arises from a gap in the macro package's design.

It is designed to output footnotes in multi-column mode at the bottom of the column, not the page.

But what to do when the document returns to single-column mode with a footnote pending?

The semantic gap is easier to imagine if we consider the hypothetical of a multi-column document formatting output in a column other the first, with a footnote pending for that column, when the input declares `1c`.  What should happen?  Should the footnote be emitted mid-page under the relevant column?

One simple possibility would be for `1c` to force a page break if any footnotes are pending.  I believe this can be checked by testing for the existence if the `|f` diversion.  (Maybe not portably, though--`.if d` is a groffism.)

Also, when a trap is set in multi-column mode for placement of the footnote, that trap is not cleared after the footnote is rendered.  The result is that the bottoms of your columns creep up the page as you go from left to right.  An example of this can be seen in bug #61675.

I am soliciting opinions as to what `1c` should mean when footnotes are pending--force a page break, or try to do something more complicated?


G. Branden Robinson <gbranden>
Group administrator
Tue 07 Dec 2021 10:55:50 PM UTC, comment #21: 

comment #20:

> the functional changes relevant to this patch.


Relevant to this bug report, I meant.

Dave <barx>
Group Member
Tue 07 Dec 2021 10:46:32 PM UTC, comment #20: 

As Bjarni points out in comment #12, most of the patch is irrelevant to this bug, largely making cosmetic changes to debugging output.  And the number and distribution of these changes makes it difficult to focus on the functional changes relevant to this patch.

So I created a new patch that contains only the pertinent changes, and also incorporates the correction Bjarni points out in comment #14.  In addition to discarding a lot of Bjarni's patch, I've added a few .rr requests after some registers' useful lifetime (next_y_pos, y_pos_trap, reached_footer, nr_traps) so as not to litter the namespace.

This revised patch is purely for academic purposes, as comment #19 demonstrates that the patch is invalid.

(file #52450)

Dave <barx>
Group Member
Tue 07 Dec 2021 02:25:21 AM UTC, comment #19: 

Bjarni's comment #12 patch (with correction from comment #14) does produce correct output for the test cases in comment #0 and comment #11.

However, the new code appears to presume that there are only two columns, whereas -me's .2c macro can begin any number of columns, two merely being the default if no number is specified.

Indeed, the patched e.tmac fails in the three-column case:

A little introductory text.
.2c 4n 3
This text appears in the first column of multi-column output.
In this example, the first column is the longest.
.bc
This text appears in the second column of multi-column output.
.bc
This text appears in the third column of multi-column output.
.1c
Single-column output resumes.


In groff 1.22.4, this behaves as expected, producing for terminal output:

A little introductory text.
This text appears    This text appears    This text appears
in the first col-    in   the   second    in the third col-
umn of multi-col-    column  of multi-    umn of multi-col-
umn  output.   In    column output.       umn output.
this example, the
first  column  is
the longest.
Single-column output resumes.

With the patch applied to the current code base, terminal output is:

A little introductory text.
This text appears    This text appears    This text appears
in the first col-    in   the   second    in the third col-
umn of multi-col-    column of  multi-    umn of multi-col-
umn  output.   In    column output.       umn output.
Single-column output resumes.
first  column  is
the longest.


The logic Bjarni added to macro .1c does not easily scale to handle an arbitrary number of columns.

Branden's dread from comment #4 was well justified.

Dave <barx>
Group Member
Sat 04 Dec 2021 11:02:54 PM UTC, comment #18: 

That's what I see now, in both terminal and typeset output.  No idea how I got the results I posted in comment #11.  Fever dream?

Dave <barx>
Group Member
Sat 04 Dec 2021 07:10:07 PM UTC, comment #17: 


comment #16:

> For some reason, now when I run that comment's test case, I'm getting different (and worse) results from what I reported back in August.  And I get these same (worse) results even if I manually apply commit ab55f50a's e.tmac change to my 1.22.4 installation, so it's not some subsequent change confounding things.  (There has been only one subsequent change to tmac/e.tmac itself, and it changed only comments.)


I definitely get something worse, with text being overprinted.


$ tgu -me EXPERIMENTS/dave-58736.me | cat -s

A little introductory text.
This  text  appears  in  the    This  text  appears  in  the
first column  of  two‐column    second column of  two‐column
output.   In  this  example,    output.
Single‐columnuoutput resumes.
than the second.


G. Branden Robinson <gbranden>
Group administrator
Sat 04 Dec 2021 04:55:45 PM UTC, comment #16: 

Did anyone replicate my results from comment #11?

...'Cause I no longer can.

For some reason, now when I run that comment's test case, I'm getting different (and worse) results from what I reported back in August.  And I get these same (worse) results even if I manually apply commit ab55f50a's e.tmac change to my 1.22.4 installation, so it's not some subsequent change confounding things.  (There has been only one subsequent change to tmac/e.tmac itself, and it changed only comments.)

In any case, it's pretty clear commit ab55f50a should be reverted, as it breaks basic (non-footnoted) two-column output -- even if I can't quite agree with my past self on exactly what the broken output is.

I still need to test Bjarni's patch from comment #12.

Dave <barx>
Group Member
Mon 23 Aug 2021 09:52:32 AM UTC, comment #15: 

comment #13:

> Anyone want to contribute some regression tests with minimal input,


Are the examples from comment #0 and comment #11 not suitable?  I could probably minimize them further if that's your only objection to them.

Dave <barx>
Group Member
Mon 23 Aug 2021 02:22:32 AM UTC, comment #14: 

  The line correction for "next_y_pos" for the first case (second
column is longer than the first one) needs to be "-1v" not "+1v".

Bjarni Ingi Gislason <bjarniig>
Mon 23 Aug 2021 01:13:52 AM UTC, comment #13: 

Anyone want to contribute some regression tests with minimal input, modeled on the existing stuff I added?


tmac/tests/e_columns-work-on-long-pages.sh
tmac/tests/e_footnotes_work_with_columns.sh


(Hmm, looks like I failed to apply my own naming convention in the latter.  Will fix.)

They don't need to be in separate scripts; separate cases within the existing script is fine.

In fact I'm happy to do the integration work.  I just want there to be clarity on what cases are being fixed.

G. Branden Robinson <gbranden>
Group administrator
Mon 23 Aug 2021 12:50:34 AM UTC, comment #12: 

  The macro "1c" just moves the vertical output place to where the
outputted text last stopped.

  It should move it to the bottom of the longer column.

  The patch in the attachment is relative to the master version.

  It hopfully fixes bug #58736.

  It could also contain some changes for bug #58447 and bug #55081.

  Most of the patch are lines with a diagnostic message.

  The actual fix is in the macros "1c" and "bc".


(file #51804)

Bjarni Ingi Gislason <bjarniig>
Fri 20 Aug 2021 07:48:31 PM UTC, comment #11: 

This change, unfortunately, breaks the -me .1c macro if the example is slightly modified from the one originally posted, even in the case without footnotes:


A little introductory text.
.2c
This text appears in the first column of two-column output.
In this example, the first column is longer than the second.
.bc
This text appears in the second column of two-column output.
.1c
Single-column output resumes.


The expected terminal output (and the actual output, before ab55f50af0599028bf7d3fd0fc50358471595b3c):


A little introductory text.
This  text  appears  in  the    This  text  appears  in  the
first column  of  two-column    second column of  two-column
output.   In  this  example,    output.
the first column  is  longer
than the second.
Single-column output resumes.


The output after this commit:


A little introductory text.
This  text  appears  in  the    This  text  appears  in  the
first column  of  two-column    second column of  two-column
output.   In  this  example,    output.
the first column  is  longer    Single-column output resumes.
than the second.


PostScript output is incorrect in the same way: text following .1c, which ought to be the full width of the page, is output in column 2.

Dave <barx>
Group Member
Fri 13 Aug 2021 02:08:32 PM UTC, comment #10: 


commit ab55f50af0599028bf7d3fd0fc50358471595b3c
Author: G. Branden Robinson <g.branden.robinson@gmail.com>
Date:   Fri Aug 13 09:32:31 2021 +1000

    [me]: Fix Savannah #58736.

    * tmac/e.tmac (1c): Fix problem when combining multiple columns,
      footnotes, and return to single-column mode: do not place the
      single-column text below the footnote.  Space to the end of the page
      (\n[nl]), not the page "high-water mark" (\n[.h]).  Thanks to Bjarni
      Ingi Gislason for the suggestion.

    Fixes <https://savannah.gnu.org/bugs/?58736>.


G. Branden Robinson <gbranden>
Group administrator
Thu 12 Aug 2021 10:56:35 PM UTC, comment #9: 

comment #8:

>   A possible fix in macro "1c" is NOT to move to the lowest place of
> the page (register .h) but to the last output line + 1 line (register
> nl + 1v).
>


Thanks, Bjarni.  The extra line doesn't appear to be necessary.

The following patch gets Dave's example working.


diff --git a/tmac/e.tmac b/tmac/e.tmac
index 00536252..5b395881 100644
--- a/tmac/e.tmac
+++ b/tmac/e.tmac
@@ -954,7 +954,7 @@
 .nr $c 1
 .nr $m 1
 .ll \\n(_Lu\"                  return to normal output
-.sp |\\n(.hu
+.sp |\\n(nlu
 .@O \\n(_ou
 ..
 .


G. Branden Robinson <gbranden>
Group administrator
Mon 09 Aug 2021 11:19:38 PM UTC, comment #8: 

  A possible fix in macro "1c" is NOT to move to the lowest place of
the page (register .h) but to the last output line + 1 line (register
nl + 1v).

Bjarni Ingi Gislason <bjarniig>
Fri 18 Sep 2020 06:22:55 PM UTC, comment #7: 

comment #3:

> If there's a repository of these versions of the -me package
> as they existed before forking, I can't find it.


Following Branden's advice, I posted a query (http://minnie.tuhs.org/pipermail/tuhs/2020-September/022175.html) to TUHS and got an almost immediate reply with a URL (http://svnweb.freebsd.org/csrg/share/me/tmac.orig_me?view=log) to a repository of -me versions 1.1 to 2.37.  (There is also a "version 8.1," but it is identical to 2.37 other than a couple comments in the header.)

Version 2.31, the basis for groff's fork, can be found at http://svnweb.freebsd.org/csrg/share/me/tmac.orig_me?revision=34394.  The earliest recorded groff version of this file (http://git.savannah.gnu.org/cgit/groff.git/tree/macros/tmac.e?id=351da0dcdf702cf243d26ffa998961bce2aa8653) has substantial changes from this v2.31.  (This may be of academic interest, but is probably not relevant to this bug, since comment #3 suggests the bug is in historical -me code as far back as v2.14.)

Dave <barx>
Group Member
Mon 20 Jul 2020 10:40:37 AM UTC, comment #6: 

Replying to myself:

comment #5:

> Savannah unfortunately requires -me bugs be categorized under
> "Macro - others," so it's hard to tell how many are filed here.
> I've opened at least eight


I realize my prolific bug-opening is skewing these results, but for comparison, currently mm has 8 open bugs, ms 4, man 7, mdoc 6, and mom 1.  By those numbers, adding a specific -me category to savannah seems justified.  -me is also the only full-service macro package lacking its own category.

> ones like this one and bug #58447, where merely describing how
> to trigger them requires an essay.


Perhaps a better thing for the -me documentation to include than complete descriptions of these bugs is links to the savannah bug reports still open at the time of the release.

Dave <barx>
Group Member
Sun 19 Jul 2020 03:50:18 PM UTC, comment #5: 

"Soon" is not among my expectations when opening groff bugs (I've opened bugs that have garnered no response for years), especially ones in macro packages that are essentially abandonware.

My intent is to put bugs where they can be prioritized and addressed when it makes sense.  Something like bug #57616, tripped in core groff just from using certain words in running text in the wrong place, should have higher priority than this one tripped by a particular combination of macros in a little-used macro package.  But the two require different areas of expertise to debug, so it's not that straightforward.

Of course, the expertise gap in -me applies to all (known and yet-to-be-discovered) -me bugs.

Savannah unfortunately requires -me bugs be categorized under "Macro - others," so it's hard to tell how many are filed here.  I've opened at least eight, with bug #58447 having the most severe result, though probably also being among the least likely to be encountered.  I try to always put "me" at the start of the summary line, though I see I've been inconsistent about using "me:", "me macros:", and "-me macros:", and forgot it altogether at least once.

> Eric Allman occasionally speaks up on the TUHS mailing list, but he may
> consider himself retired--perhaps LONG retired--from maintaining me(7).


Yeah, if someone asked me to debug code I hadn't looked at in over 30 years, my first thought would be that a person who's never seen the code before and I would be starting in about the same place.  But I don't see any down side to asking.

And, true, aside from Eric, there may not be a good candidate for troubleshooting -me problems.  I find the -me code impenetrable.  A few email-list regulars (Tadziu, Ingo, Ralph) could probably figure them out if they had the time and motivation.  George Helffrich seems to have some expertise, in that he submitted a patch for -me bug #57638--albeit a bug he introduced three years earlier, which speaks to the danger of not having regression tests.

Still, without downplaying this danger, on the other hand I'd point out:

  • While we don't have a sizeable -me corpus like we do with -man, the four doc/*.me documents in groff's source tree should catch any blatant problems.
  • Given the age of the original macro set and the software-development methodology widely used at the time, the original development probably included limited or no regression testing, which could be responsible for the existence of many of these current bugs.  So continuing to develop without this safety net, fixing known bugs with the potential of inserting new ones as a side effect, doesn't strike me as a step backwards.
  • In fact I'd hazard that modern revision control will make any regressions discovered in the future easier to pinpoint (as I did in 57638); it's bugs in the historical -me code that are the least tractable.


But until someone volunteers to take on these risks and challenges, perhaps documenting all these bugs is best--though even that isn't easy for ones like this one and bug #58447, where merely describing how to trigger them requires an essay.  And it's tricky with ones like bug #55060 and bug #58682, where the very problem is a discrepancy between documentation and functionality with it being unclear which should be considered correct.

Dave <barx>
Group Member
Sat 18 Jul 2020 11:14:40 PM UTC, comment #4: 

Thanks for doing the research on this, Dave.

Hacking full-service macro packages like ms, mm, and me without any test suite for them, or even a corpus of example documents we can regression-test with (like man) fills with dread, and little hope we can resolve this soon.

Think this might be worth documenting in a "Bugs" (sub)section of groff_me(7)?  Maybe in meintro.me and meref.me as well.  That wouldn't enable us to close this bug, but it would serve as warning to interested users and might get a little more light thrown on the issue.

Eric Allman occasionally speaks up on the TUHS mailing list, but he may consider himself retired--perhaps LONG retired--from maintaining me(7).

Thoughts?

G. Branden Robinson <gbranden>
Group administrator
Sat 18 Jul 2020 07:35:06 PM UTC, comment #3: 

The footnoted input file produces the same incorrect PostScript output in Heirloom troff, and the unfootnoted one produces the same correct PostScript output.  (Heirloom nroff output from both input files is pretty hopelessly broken.)

According to the header blocks of the respective files, groff's -me macros are forked from version 2.31 of Eric Allman's original set, and Heirloom's from version 2.14.  If there's a repository of these versions of the -me package as they existed before forking, I can't find it.

Dave <barx>
Group Member
Sat 18 Jul 2020 04:04:50 AM UTC, comment #2: 

I guess a reasonable next step is to find out if the me macros misbehave similarly elsewhere, or if this problem arose due to groff (either our fork of me(7) or something within the typesetting logic itself).

G. Branden Robinson <gbranden>
Group administrator
Fri 17 Jul 2020 03:02:30 AM UTC, comment #1: 

Can confirm this.

G. Branden Robinson <gbranden>
Group administrator
Wed 08 Jul 2020 05:40:28 PM UTC, original submission:  

This bug shows up in both PostScript and terminal output.  The example below uses nroff because that's easier to show in a bug report.  It happens on a recent groff build from git and in official groff release 1.22.3.

This input file:


A little introductory text.
.2c
This text appears in the first column of two-column output.
.bc
This text appears in the second column of two-column output.
.1c
Single-column output resumes.


when run with "nroff -me", produces:


A little introductory text.
This  text  appears  in  the    This  text  appears  in  the
first column  of  two-column    second column of  two-column
output.                         output.
Single-column output resumes.


(plus a lot of blank lines filling out the "page").  Now change the first line of the input file to these four lines:


A little introductory text.*
.(f
*A footnote.
.)f


Now, the line "Single-column output resumes," rather than appearing below the two columns as it should, is pushed to the bottom of the page, below the footnote.  It looks basically like this (though I've greatly reduced the number of blank lines):


A little introductory text.*
This  text  appears  in  the    This  text  appears  in  the
first column  of  two-column    second column of  two-column
output.                         output.











____________________
   *A footnote.
Single-column output resumes.


Dave <barx>
Group Member

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #53755:  58736.diff added by gbranden (3KiB - text/x-patch)
file #52450:  58736.e.tmac.patch added by barx (1KiB - text/x-diff - streamlined version of previous patch)
file #51804:  58736.e.tmac.diff added by bjarniig (12KiB - text/x-patch)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by bjarniig (Posted a comment)
  • -email is unavailable- added by gbranden (Posted a comment)
  • -email is unavailable- added by barx (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 18 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2022-10-03 gbranden StatusNeed Info Fixed
        Open/ClosedOpen Closed
    2022-09-26 gbranden Attached File- Added 58736.diff, #53755
        Attached File- Added footnoted-column-switcher-2.me, #53756
        Attached File- Added footnoted-column-switcher-1.me, #53757
    2021-12-15 gbranden StatusIn Progress Need Info
    2021-12-07 barx Attached File- Added 58736.e.tmac.patch, #52450
    2021-08-23 gbranden StatusFixed In Progress
    2021-08-23 bjarniig Attached File- Added 58736.e.tmac.diff, #51804
    2021-08-20 barx Open/ClosedClosed Open
    2021-08-13 gbranden StatusIn Progress Fixed
        Open/ClosedOpen Closed
        Planned ReleaseNone 1.23.0
    2021-08-12 gbranden StatusConfirmed In Progress
        Assigned toNone gbranden
    2021-06-04 barx Summary-me macros: footnote breaks two-column output [me] footnote breaks two-column output
    2020-07-25 gbranden CategoryMacro package - others/general Macro package me
    2020-07-17 gbranden StatusNone Confirmed

    Back to the top

    Powered by Savane 3.13-4b48.
    Corresponding source code