bugGNU roff - Bugs: bug #61734, [man] localize like other packages

 
 

bug #61734: [man] localize like other packages

Submitter:  G. Branden Robinson <gbranden>
Submitted:  Tue 28 Dec 2021 10:50:35 AM UTC
   
 
Category:  Macro man Severity:  3 - Normal
Item Group:  Feature change Status:  Fixed
Privacy:  Public Assigned to:  gbranden
Open/Closed:  Closed Planned Release:  1.23.0
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Wed 15 Jun 2022 03:35:46 PM UTC, comment #11: 

Restoring status to "Fixed" since the me(7) issue was spun off into bug #62181 (and resolved).

G. Branden Robinson <gbranden>
Group administrator
Mon 14 Mar 2022 09:37:06 PM UTC, comment #10: 

I harped enough on how the -me issue should be a new bug report (I mentioned it in comment #7, then forgot I did so and said it again in comment #9) that Branden opened one: bug #62181.

Dave <barx>
Group Member
Sat 12 Mar 2022 01:36:48 AM UTC, comment #9: 

I too have muddied the waters by bringing up separate issues without clearly delineating them.

To frame the discussion, I'd say the places groff's e.tmac needs to work, in order of importance, are

  1. within groff
  2. in legacy troff
  3. in other troffs


I suspect Heirloom has the largest market share of the third category, but, as you note, is also the most problematic in terms of implementing noncompatible extensions.

So, taking these issues in my proposed order:

  1. Groff itself will misbehave here if \\n[\\*[locale]*hyphenation-mode-trap] is undefined.  Maybe with a stock installation, this is impossible (I haven't tried to find a way to induce it), but surely a user can define new locales or hack groff's initialization files locally, in such a way to leave the localization register undefined.  It doesn't seem like defaulting .hy to 0 rather than 6 in these instances is right.
  2. This is where we seem to be on solidest ground: as you note, legacy troff won't know a .do request, and will thus ignore that line.
  3. This may, ultimately, be an unwinnable game of whack-a-mole, since existing troff implementations can always change, or new ones be created, over time.  But a starting point might be to test whether the .g register is defined.  This will surely break at least Heirloom in its not-actually-compatible "groff compatibility mode," but I'm willing to call that an Heirloom bug: don't claim groff compatibility if you don't actually offer it.


Let me know if I've gone astray anywhere.  (If there end up being further changes to be made, it probably warrants a new bug report: this one only concerned the -man package, though the resolving commit did try to sneak -me in the door as a +1.)

Dave <barx>
Group Member
Sat 05 Mar 2022 02:38:59 PM UTC, comment #8: 

Hi, Dave!

comment #7:

> I believe you that this is fixed for real in -man, but the -me change does not seem entirely valid.  The new code added:


> .\" Set package default hyphenation mode, but override it with groff's
> .\" localized value if available.
> .hy 6
> .do hy \\n[\\*[locale]*hyphenation-mode-trap]


> says in the comment that it conditionally uses groff's localization if available, but in the code unconditionally overwrites the ".hy 6" with whatever is in the localization register--which, if it doesn't exist, is interpolated as 0, almost certainly not what is intended.


My reasoning here is that a legacy troff will ignore the `do` request completely since it won't be a request or the name of a defined macro.

However, I didn't test this change against any other troff implementation, so it's possible my reasoning is wrong.

Are you testing with Heirloom Doctools troff?  I just now checked their doc.ps document--their 78-page expanded version of CSTR #54--and don't find myself much the wiser.  It says if a `cp` request is encountered, a bunch of groff compatibility features are turned on no matter what.  `.cp 0` simply turns on more of them.  `.cp 1` on the other hand, apparently, does not get Heirloom out of groff compatibility mode altogether.

On the gripping hand, groff e.tmac does not issue a `cp` request.

If I understand Heirloom's manual correctly, then to simulate groff's compatibility mode, you have to issue `do xflag 0`.

> The straightforward solution for groff is to sequester the second .hy call behind an ".if r" on the register.
>
> This breaks the package's compatibility with other *roffs, which probably lack the r conditional (even Heirloom, which has numerous groff extensions, lacks it), torpedoing one of the stated reasons for the dual .hy setting.  There may be some clever way I'm not thinking of to preserve that compatibility, but at least the groff behavior should be correct in the case where no localization register is set.


I infer that `do` was deliberately engineered by groff to perform such sequestration, and it appears to me that Heirloom has muddied the waters by giving it different default semantics.

> "[Extension/xflag level] 1 enables extensions except for direct access to long names, i.e., .abcde will be interpreted as request ab with argument cde and \*[xyz] refers to the string named [ followed by the text xyz].  Long names can be accessed using the do request, e.g., .do de abcde, .do if 1 \*[xyz].  This level is the default.


(In other news, Savane's "rich text" markup seems to be kind of crap.  I can't figure out how to set "\*[xyz]" in bold.  If a word starts with '*', the next '*' turns it off, without any way to escape it or to add a zero-width break that looks like a word boundary to it.  Maybe it's possible, but the "Rich Markup" help page offers no clues.)

So, that's kind of frustrating.  I'm deliberately trying to accommodate legacy troff users and Heirloom is getting the way with this complex feature that syntactically claims groff compatibility (also see how they set the .g register) without being willing to support all of groff's features.

> This maybe should be a new bug report, since the current one is primarily concerned with -man, but I thought I'd float it here first since this is where the commit was registered.


First let's confirm if I'm understanding the situation correctly!

G. Branden Robinson <gbranden>
Group administrator
Fri 04 Mar 2022 06:53:49 PM UTC, comment #7: 

I believe you that this is fixed for real in -man, but the -me change does not seem entirely valid.  The new code added:

.\" Set package default hyphenation mode, but override it with groff's
.\" localized value if available.
.hy 6
.do hy \\n[\\*[locale]*hyphenation-mode-trap]

says in the comment that it conditionally uses groff's localization if available, but in the code unconditionally overwrites the ".hy 6" with whatever is in the localization register--which, if it doesn't exist, is interpolated as 0, almost certainly not what is intended.

The straightforward solution for groff is to sequester the second .hy call behind an ".if r" on the register.

This breaks the package's compatibility with other *roffs, which probably lack the r conditional (even Heirloom, which has numerous groff extensions, lacks it), torpedoing one of the stated reasons for the dual .hy setting.  There may be some clever way I'm not thinking of to preserve that compatibility, but at least the groff behavior should be correct in the case where no localization register is set.

This maybe should be a new bug report, since the current one is primarily concerned with -man, but I thought I'd float it here first since this is where the commit was registered.

Dave <barx>
Group Member
Wed 02 Mar 2022 10:13:57 AM UTC, comment #6: 


commit 254a2cc54e685289d460ddc50090fecc2ad291f0
Author: G. Branden Robinson <g.branden.robinson@gmail.com>
Date:   Tue Feb 22 22:42:29 2022 +1100

    [man]: Regression-test Savannah #61734.

    * tmac/tests/an_reset-hyphenation-correctly.sh: Do it.
    * tmac/tmac.am (tmac_TESTS): Run test.

    Test fails at this commit.

commit 920fff1cf59d38bacd9b1b99b3d1ce3ce4e1e13f
Author: G. Branden Robinson <g.branden.robinson@gmail.com>
Date:   Wed Feb 23 00:55:29 2022 +1100

    [man,mdoc,me]: Revise hyphenation localization.

    * tmac/an.tmac (an*reset-hyphenation-mode): New macro sets the
      hyphenation mode appropriate to the locale depending on the values of
      `cR` (continuous rendering mode) and `HY` (hyphenation enablement);
      the latter is now interpreted as a Boolean.  Define new register
      `an*hyphenation-mode` reflecting these conditions.

      (TH): Call the foregoing instead of invoking `hy` request.

      (MR): Restore hyphenation mode to `an*hyphenation-mode`.

      (initializtion): Simplify; if not defined, set `HY` to 1.

    * tmac/e.tmac (ld): Set the hyphenation mode here instead of in
      initialization (which already calls this macro).  Set the mode to 6
      for the benefit of anyone borrowing our me(7) changes, but then
      immediately change to the mode determinated by the localization file.

    * tmac/mdoc/doc-common (doc-hyphen-flags): Set an hyphenation mode
      appropriate to the locale depending on the values of `cR` and `HY`.
    * tmac/mdoc/doc-ditroff:
    * tmac/mdoc/doc-nroff: Simplify initialization; if not defined, set `HY`
      to 1.

    * tmac/cs.tmac:
    * tmac/de.tmac:
    * tmac/en.tmac:
    * tmac/fr.tmac:
    * tmac/it.tmac:
    * tmac/sv.tmac: Before switching locales, set a flag if troublesome bit
      2 of the hyphenation mode is set so that we can preserve it.  Shift
      man(7) and me(7) logic to follow setup of the new hyphenation mode;
      for the former, call the new `an*reset-hyphenation-mode`.

    * tmac/groff_man.7.man.in:
    * tmac/groff_mdoc.7.man: Document updated meaning of `HY` register.

    * NEWS: Add item noting change of man(7) `HY`'s meaning.  Update
      existing item regarding mdoc(7)'s support for it (groff 1.22.4 mdoc(7)
      didn't recognize the register at all).

    Fixes <https://savannah.gnu.org/bugs/?61734> (for real this time?).


G. Branden Robinson <gbranden>
Group administrator
Sat 05 Feb 2022 05:30:30 PM UTC, comment #5: 

I had noticed that the Swedish hyphenation mode was sticking to
our English pages after groff_mmse(7) in groff-man-pages.pdf.

This is proving to be quite the devil to deal with, thanks to
the large number of constraints imposed by batch rendering of
multiple pages (which may be in different languages), the
wrapping of the macro package itself by andoc.tmac (because man
pages can be written with different *roff macro packages),
the user-customizability of the hyphenation mode, and that
damned bit in the hyphenation mode that has **-all to do with
the hyphenation patterns but instead is a knob for suppressing
hyphenation when a page location trap is about to spring.

I'm including a diff of me trying to solve the problem within
all of these constraints, and it works for our use case, but I
am not happy at all with the level of complexity it demands.

I propose instead to revise the semantics of the man(7) `HY`
macro and make it a Boolean.  Hyphenation on or off--if on, you
get what our macro files decide for you.  Does any user really
want to decide the details of the hyphenation mode themselves?

I have a pretty strong suspicion that all people use `HY` for in
the first place is mashing a big "SHUT HYPHENATION OFF" button
anyway (and that's what man-db man(1)'s --nh option already
does for the 99% who use `man` instead of `groff -man`).

(Or mandoc(1), but it doesn't affect the decision here because
as I understand it, it doesn't support automatic hyphenation and
Ingo doesn't plan to ever add it.)

If someone really wants to seize control of the hyphenation
mode, they're free to redefine the `hy` request in man.local.

I invite them to keep all the pieces.

Next steps:

I need to write a regression test to catch the case I observed
(the hyphenation in question was a ghastly "g-lyph").

Then implement `HY` boolification, and see how much less complex
that route really is.

The heart of this is probably, in `TH`, turning


.  hy \\n[HY]


into


.  if \\n[HY] .hy \\n[an*hyphenation-mode]


or maybe


.  ie \\n[HY] .hy \\n[an*hyphenation-mode]
.  el         .hy 0


To support recovery from rogue pages that sneakily turned it on.

Note for anyone reading hastily: the attached commit is not
pushed as of this writing.

(file #52808)

G. Branden Robinson <gbranden>
Group administrator
Thu 03 Feb 2022 10:31:52 PM UTC, comment #4: 

Nope, we're still not quite there.

G. Branden Robinson <gbranden>
Group administrator
Fri 28 Jan 2022 06:37:28 PM UTC, comment #3: 

Re-re-fixed.


commit c859eafcd8cd50af40b9d6aa402ec591c434964b
Author: G. Branden Robinson <g.branden.robinson@gmail.com>
Date:   Tue Jan 25 22:15:07 2022 +1100

    tmac/trans.tmac: More commit c64fd60dff follow-up.

    * tmac/trans.tmac: Further follow up commit c64fd60dff.  If the `andoc`
      wrapper is being used, perform man(7) localization (not just when `an`
      is loaded).


G. Branden Robinson <gbranden>
Group administrator
Mon 24 Jan 2022 04:59:00 PM UTC, comment #2: 

It turns out this wasn't done.


commit 57b3e1074f01b5a6cebc99461db1c19cb6720071
Author: G. Branden Robinson <g.branden.robinson@gmail.com>
Date:   Mon Jan 24 20:21:47 2022 +1100

    [man]: Fix manual section title localization.

    Commit c64fd60dff was not quite cooked.  The localized strings (in
    English) for the manual section titles need to be set at every document
    load (i.e., when `TH` is called), but only if the groff locale is
    "english".  When switching locales, these strings were getting "stuck"
    in their non-English translations even when returning to an English
    page.

    * tmac/an.tmac (an*localize): New macro initializes strings.

      (TH): Call `an*localize` if the locale is "english".

      (initialization): Drop initialization of section title strings.

    Continues fixing <https://savannah.gnu.org/bugs/?61734>.

    Tested with several scenarios, including this one that starts out with a
    non-English page and then switches back and forth between languages.
    The package is not tripped up and the center header remains appropriate.

    $ ./build/test-groff -man -msv EXPERIMENTS/swedish.1 \
      tmac/en.tmac EXPERIMENTS/english.1 \
      tmac/sv.tmac EXPERIMENTS/swedish.1 \
      tmac/en.tmac EXPERIMENTS/english.1


...and even after that it still wasn't done, because I forgot about
andoc.tmac.

Here's a further patch, pending for my next push.


diff --git a/tmac/trans.tmac b/tmac/trans.tmac
index d5257f52a..a06426b3e 100644
--- a/tmac/trans.tmac
+++ b/tmac/trans.tmac
@@ -31,7 +31,14 @@
 .\" trans.tmac should stay locale-independent.
 .
 .\" man package
-.if d an \{\
+.\"
+.\" Either an.tmac or andoc.tmac may already be loaded.
+.nr trans*is-man 0
+.
+.if d an         .nr trans*is-man 1
+.if d reload-man .nr trans*is-man 1
+.
+.if \n[trans*is-man] \{\
 .      als an*section1 \*[locale]-man-section1
 .      als an*section2 \*[locale]-man-section2
 .      als an*section3 \*[locale]-man-section3
@@ -43,6 +50,8 @@
 .      als an*section9 \*[locale]-man-section9
 .\}
 .
+.rr trans*is-man
+.
 .\" ms package
 .if r GS \{\
 .      als ABSTRACT \*[locale]-abstract


G. Branden Robinson <gbranden>
Group administrator
Tue 11 Jan 2022 11:36:00 AM UTC, comment #1: 


commit c64fd60dff62f5dbc6271f8fbc538129a28e1fdd
Author: G. Branden Robinson <g.branden.robinson@gmail.com>
Date:   Tue Jan 4 06:37:34 2022 +1100

    [man]: Internationalize and localize.

    * tmac/an.tmac (TH): Indirect predefined manual section titles through
      new strings `an*section[1-9]` so that they can be localized.

      (initialization): Give these strings English defaults.

      (TH): Move `ss` request from here...

    * tmac/en.tmac: ...to here.  It's not the man(7) package's business what
      the inter-word and inter-sentence spacing values are.  Also, having
      the correct[disputed] inter-sentence spacing value for English in its
      localization file restores it for the benefit of multi-lingual
      documents (mainly roff applications other than man(7)).

    * tmac/{cs,de,fr,it,sv}.tmac: Supply translations for the manual section
      titles; these were done by a monoglot American employing Google
      Translate, so those scandalized or running into howlingly wrong
      locutions are warmly invited to submit corrections.

    * tmac/trans.tmac: Load localized strings.

    Fixes <https://savannah.gnu.org/bugs/?61734>.


G. Branden Robinson <gbranden>
Group administrator
Tue 28 Dec 2021 10:50:35 AM UTC, original submission:  

The only localizable strings in the man(7) package are those
used for the sections of the manual (1 through 8).

Translations for these in more languages than we currently
otherwise support are likely available in Helge Kreutzmann's
manpages-l10n project.

G. Branden Robinson <gbranden>
Group administrator

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #52808:  61734-dubious.diff added by gbranden (15KiB - text/x-patch)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by barx (Posted a comment)
  • -email is unavailable- added by gbranden (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 16 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2022-06-15 gbranden StatusNeed Info Fixed
    2022-03-05 gbranden StatusFixed Need Info
    2022-03-02 gbranden StatusIn Progress Fixed
        Open/ClosedOpen Closed
    2022-02-05 gbranden Attached File- Added 61734-dubious.diff, #52808
    2022-02-03 gbranden StatusFixed In Progress
        Open/ClosedClosed Open
    2022-01-28 gbranden StatusIn Progress Fixed
        Open/ClosedOpen Closed
    2022-01-24 gbranden StatusFixed In Progress
        Open/ClosedClosed Open
    2022-01-22 gbranden Item GroupNone Feature change
    2022-01-11 gbranden StatusIn Progress Fixed
        Open/ClosedOpen Closed
        Planned ReleaseNone 1.23.0
    2022-01-05 gbranden StatusNone In Progress

    Back to the top

    Powered by Savane 3.13-f8d8.
    Corresponding source code