bugGNU roff - Bugs: bug #55155, [troff] .tr is applied to the RHS...

 
 

bug #55155: [troff] .tr is applied to the RHS of .char definitions

Submitter:  Dave <barx>
Submitted:  Mon 03 Dec 2018 05:50:22 PM UTC
   
 
Category:  Core Severity:  3 - Normal
Item Group:  Incorrect behaviour Status:  Confirmed
Privacy:  Public Assigned to:  gbranden
Open/Closed:  Open Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Thu 28 Dec 2023 01:18:29 AM UTC, comment #17: 

comment #10:

> Reassigning to self and keeping in "Need Info" status while
> I take this to the list.


Said list discussion petered out months ago (summarized in comment #11), so this ticket should no longer be waiting on feedback.  Updating status.

Dave <barx>
Group Member
Wed 12 Apr 2023 11:32:24 PM UTC, comment #16: 

comment #15:

>   So, it is the responsibility of the writer to ensure that the
> transliteration is reversed in time.


It's not clear that this should be the writer's responsibility.  Even were it documented as such (it currently is not), it would still feel like pushing an unreasonable requirement onto the writer (albeit not as unreasonable as the one in "The documented problem" section of bug #58562) that software ought to be able to handle.

But even if we accept your premise...

>   Ergo: no need to mess with the code of the request "tr".


...this conclusion does not follow from it:

  • There are other situations (e.g., comment #4) where the current behavior is not ideal.
  • The informal email survey about the matter did not even cite the "Typesetting Mathematics" situation, yet its respondents favored the change 3 to 1 (see comment #11).
Dave <barx>
Group Member
Wed 12 Apr 2023 09:56:15 PM UTC, comment #15: 


comment #13:

>
> comment #12:
> > Further musings:
> >
> > All the example usages in the email thread appeared to be contrived for this thread; no one responded with a clear real-world case that would be affected by the change one way or the other.
>
> I can address that.
>
> Format Kernighan & Cherry's "Typesetting Mathematics -- User's Guide (Second Edition)" for PostScript or PDF and watch the page header bounce back and forth between hyphens and minus signs.
>
> Procedure:
>
> 1. Obtain the paper's source files.  https://minnie.tuhs.org/cgi-bin/utree.pl?file=V7/usr/doc/eqn/
> 2. Apply the attached patch.
> 3. make eqnuser.pdf
> 4. Compare, e.g., the page headers on pages 2 and 3.
>
> Optional step 2a--review the patch file and verify that there is nothing up my sleeve.  The sources of the user's guide are the files starting with "g".
>
> (file #54594)


  This is a bug in the input source.  The transliteration is absent if
the formatting is for an A4 page (groff ... -dpaper=A4 ...)

  The cause of the visibility of the bug is a page break between the
macros P1 and P2 in file "g1".

  Adding a test before macro P1:

.\" Added to avoid a page brake before .P2
.if \n(.t<4v .bp
.P1
x=y+z
.P2

resolves the problem.

  So, it is the responsibility of the writer to ensure that the
transliteration is reversed in time.

  Ergo: no need to mess with the code of the request "tr".

Bjarni Ingi Gislason <bjarniig>
Mon 10 Apr 2023 05:44:22 PM UTC, comment #14: 

comment #13:

> Kernighan & Cherry's "Typesetting Mathematics -- User's Guide (Second Edition)"


Ah, this must be the mysterious "historical troff document" you couldn't recall in http://lists.gnu.org/r/groff/2023-04/msg00004.html.  From the description of the problem there, it sounded to me more like bug #62691's purview than this one's, but I guess they're related insofar as they both entail more precisely delineating .tr's dominion.

Dave <barx>
Group Member
Mon 10 Apr 2023 05:08:29 PM UTC, comment #13: 


comment #12:

> Further musings:
>
> All the example usages in the email thread appeared to be contrived for this thread; no one responded with a clear real-world case that would be affected by the change one way or the other.


I can address that.

Format Kernighan & Cherry's "Typesetting Mathematics -- User's Guide (Second Edition)" for PostScript or PDF and watch the page header bounce back and forth between hyphens and minus signs.

Procedure:

1. Obtain the paper's source files.  https://minnie.tuhs.org/cgi-bin/utree.pl?file=V7/usr/doc/eqn/
2. Apply the attached patch.
3. make eqnuser.pdf
4. Compare, e.g., the page headers on pages 2 and 3.

Optional step 2a--review the patch file and verify that there is nothing up my sleeve.  The sources of the user's guide are the files starting with "g".

(file #54594)

G. Branden Robinson <gbranden>
Group administrator
Mon 10 Apr 2023 09:58:03 AM UTC, comment #12: 

Further musings:

All the example usages in the email thread appeared to be contrived for this thread; no one responded with a clear real-world case that would be affected by the change one way or the other.

While most of the presented cases would give improved output with the change, a DWIM reading of the code in Ralph's example does argue for the current behavior.  But I can think of no way to detect that intention algorithmically, and any attempt to guess the user's intent will doubtless make the behavior harder to document.  So I think this will have to be a unilateral behavior, which (whichever way it's decided) will make some things behave badly.  (The roff language already makes few if any concessions to the DWIM philosophy.)

Dave <barx>
Group Member
Mon 10 Apr 2023 09:31:05 AM UTC, comment #11: 

Arguments against the change:


Arguments for the change:


Discussion seems to have ended with no further responses for 6 days.

Dave <barx>
Group Member
Fri 31 Mar 2023 04:11:12 PM UTC, comment #10: 

comment #9:

> comment #8:
> > I would therefore propose changing this behavior, such that `tr`
> > does not apply to the RHS of `char` character definitions.
>
> I do feel like this is more in line with this sentence from the info manual: "the first argument of 'tr' should be an input character or entity, and the second one a glyph entity."  (These words remain unchanged since being added in 2002, commit 21260e1c.)  Although the docs are not explicit about this, I think intuitively the RHS of a .char definition wouldn't be considered "an input character or entity."


Agreed.

> > If you agree, then we should shop this change to the groff list
> > to see if it draws protest from anyone relying on this behavior.
>
> Yeah, some groffers do exploit obscure behavior.  On the plus side, .char and its family originate with groff (though at least Heirloom also implements them),


Yeah.  WRONGLY.  (I mean groff-incompatibly in this case, but that's a good thing, as it increases our latitude for just fixing this.)


$ cat ATTIC/zz_top.groff
.tr zx
.char \(zz zeezee
\(zz top
.pl \n(nlu
$ ~/heirloom/bin/nroff ATTIC/zz_top.groff | cat -s
 top


As you can perhaps see, this input also screwed up Heirloom troff's ability to interpret the (portable!) `pl` request.  Something to do with having the input line undergo character translation, expansion of a character definition, or both, confused it.

> so we don't have to worry about deviating from AT&T.


Reassigning to self and keeping in "Need Info" status while I take this to the list.

G. Branden Robinson <gbranden>
Group administrator
Fri 31 Mar 2023 03:47:39 PM UTC, comment #9: 

comment #8:

> I would therefore propose changing this behavior, such that `tr`
> does not apply to the RHS of `char` character definitions.


I do feel like this is more in line with this sentence from the info manual: "the first argument of 'tr' should be an input character or entity, and the second one a glyph entity."  (These words remain unchanged since being added in 2002, commit 21260e1c.)  Although the docs are not explicit about this, I think intuitively the RHS of a .char definition wouldn't be considered "an input character or entity."

> If you agree, then we should shop this change to the groff list
> to see if it draws protest from anyone relying on this behavior.


Yeah, some groffers do exploit obscure behavior.  On the plus side, .char and its family originate with groff (though at least Heirloom also implements them), so we don't have to worry about deviating from AT&T.

Dave <barx>
Group Member
Thu 30 Mar 2023 01:19:04 PM UTC, comment #8: 

Unfortunately for me, my mental model in comment #7 does not accurately describe groff's behavior.


$ cat ATTIC/zz_top.groff
.tr zx
.char \(zz zeezee
\(zz top
.pl \n(nlu
$ nroff ATTIC/zz_top.groff
xeexee top


You did warn me:

> The manual explicitly says a glyph defined by .char can be the source or target of a .tr


...that much is okay...

> and that .char itself is nonrecursive,


...as is this...

> but this particular interaction does not seem to be directly addressed.


...which leads directly to the summary of this ticket.

I would therefore propose changing this behavior, such that `tr` does not apply to the RHS of `char` character definitions.

If you agree, then we should shop this change to the groff list to see if it draws protest from anyone relying on this behavior.

People might not even know they're relying on this; if it's practical, it might help to add a diagnostic warning of a character appearing in a `char` family RHS also occurring as a...transliterand.

G. Branden Robinson <gbranden>
Group administrator
Thu 30 Mar 2023 12:55:41 PM UTC, comment #7: 

I'm thinking we need a stronger distinction, in documentation and possibly in code, between translations of characters (those things that are manipulated before conversion into glyph entities for the output device) and glyphs (things that the output device is expected render at a certain size using a certain font, possibly with other properties like stroke color and slant).

We might even need two more categories. :(

1. input character remapping (`trin`; handles character encoding issues from the input file stream)
2. character lookup remapping (`tr`); once a character is canonicalized into the "groff character set", you can switch them around
3. rewriting/redefinition of output character entities (`char` and friends); this determines what is produced in the "grout" commands `C`, `c`, `t`, `u`, and the legacy, "optimized" encoding described in CSTR #97.
4. what an output device produces with (virtual) ink on a (virtual) page

If this is sensible and coherent, it means I'm closer to being ready to revise the "Using Symbols" section of our Texinfo manual.

Throwing to you for feedback.

G. Branden Robinson <gbranden>
Group administrator
Mon 01 Aug 2022 04:36:21 AM UTC, comment #6: 

True.  For the cited example, the behavior seems clearly wrong.  In the general case, it may be less obvious what should happen.

.tr zx
.char \(zz zeezee
\(zz top

Is this expected to output "zeezee top" or "xeexee top"?

The manual explicitly says a glyph defined by .char can be the source or target of a .tr, and that .char itself is nonrecursive, but this particular interaction does not seem to be directly addressed.

Dave <barx>
Group Member
Tue 11 Jan 2022 02:11:32 AM UTC, comment #5: 

This is a core issue, right down deep in src/roff/troff, and is not, as I understand it, device-specific in any way.  tty-char.tmac simply makes it visible.

It might demand a good deal of thought.

I threw '.trin` instead of `.tr` at the problem, not really thinking it would help (the problem is not the encoding of input characters), and it did not.

G. Branden Robinson <gbranden>
Group administrator
Tue 11 May 2021 04:49:18 AM UTC, comment #4: 

The example command in comment #2 produces the given output only if nroff detects that the user has an ASCII or Latin-1 environment.  To generalize the example to work in any non-EBCDIC terminal, give nroff an explicit -T option:


$ printf         'A \[if] B\n' | nroff -Tascii | head -1
A <infinity> B
$ printf '.tr n-\nA \[if] B\n' | nroff -Tascii | head -1
A <i-fi-ity> B


Dave <barx>
Group Member
Tue 04 Dec 2018 11:03:25 AM UTC, comment #3: 

(That should read "commit babca15fa2d3f5ec1dd7214e31e1efc42759c220" -- copy/paste error)

Dave <barx>
Group Member
Tue 04 Dec 2018 04:25:44 AM UTC, comment #2: 

It turns out commit babca15fa2d3f5ec1dd7214e31e1efc42759c220vax solved this specific instance of the problem by changing how nroff displays a dagger symbol.  So the examples below do not fail in the manner described using the latest groff code and .tmac files.

But the general problem -- that .tr sometimes changes characters that don't actually appear in input -- remains.  Using the latest tmac/tty-char.tmac, for instance, you can get the result:

$ printf '.tr n-\nA \[if] B\n' | nroff | head -1
A <i-fi-ity> B

.tr should only change n's that appear in input, not ones that appear only in output, and especially not ones part of a predefined symbol.

Dave <barx>
Group Member
Mon 03 Dec 2018 06:04:14 PM UTC, comment #1: 

UGH, this bug tracker is so broken.  I have no idea why it munged the output of those od commands.  At least the code is unmolested, so you can generate the correct output on your own system.

Dave <barx>
Group Member
Mon 03 Dec 2018 05:50:22 PM UTC, original submission:  

$ printf 'A \[dg] B\n' | nroff | od -t x1z | head -1
0000000 41 20 7c 08 2d 20 42 0a 0a 0a 0a 0a 0a 0a 0a 0a  >A |.- B.........<

This shows that nroff outputs the dagger character as 7C 08 2D -- that is, a vertical bar, a backspace, and a hyphen -- a perfectly reasonable facsimile of a dagger in ASCII characters.

Now add a .tr request at the start of the file to change vertical bars in input to the letter O:

$ printf '.tr |O\nA \[dg] B\n' | nroff | od -t x1z | head -1
0000000 41 20 4f 08 2d 20 42 0a 0a 0a 0a 0a 0a 0a 0a 0a  >A O.- B.........<

Now the dagger is represented by 4F 08 2D (capital O, backspace, hyphen -- not at all a reasonable dagger construction).  The .tr is modifying not just vertical bars in the input (which is its documented function: "the first argument of `tr' should be an input character or entity") but a vertical bar that was part of the way nroff built the dagger.  It's difficult to imagine a situation in which this would be desirable.

Dave <barx>
Group Member

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by bjarniig (Posted a comment)
  • -email is unavailable- added by gbranden (Posted a comment)
  • -email is unavailable- added by barx (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 8 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2023-12-28 barx StatusNeed Info Confirmed
    2023-04-10 gbranden Attached File- Added hacking-typesetting-mathematics-2e-rev2.diff, #54594
    2023-03-31 gbranden Assigned tobarx gbranden
    2023-03-30 gbranden StatusConfirmed Need Info
        Assigned toNone barx
    2022-01-11 gbranden CategoryDriver - others/general Core
        StatusNone Confirmed
        Summary.tr is overzealous in nroff output [troff] .tr is applied to the RHS of .char definitions

    Back to the top

    Powered by Savane 3.13-758e.
    Corresponding source code