bugGNU roff - Bugs: bug #53196, .hy values 4 and 8 ignored for...

 
 

bug #53196: .hy values 4 and 8 ignored for hyphenation exceptions read with .hpf or .hpfa

Submitter:  Dave <barx>
Submitted:  Wed 21 Feb 2018 04:55:57 AM UTC
   
 
Category:  Core Severity:  2 - Minor
Item Group:  Incorrect behaviour Status:  Fixed
Privacy:  Public Assigned to:  wl
Open/Closed:  Closed Planned Release:  1.22.4
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Fri 04 May 2018 06:51:18 PM UTC, comment #8: 

Yes, I confirm that the core issue and the documentation issue (elucidated in comment #2) have both been fixed.  Thank you!

Dave <barx>
Group Member
Fri 02 Mar 2018 05:57:37 PM UTC, comment #7: 

The issue is now fixed in git.  Please test!

Werner LEMBERG <wl>
Group administrator
Wed 28 Feb 2018 08:55:32 PM UTC, comment #6: 

Because the regression Bjarni reported in comment #3 is a separate issue from the primary one in this bug report, I've opened bug #53243 for it.  Further comments about overall use of .hy 4 and 8 should go there.

To respond to comment #5:

I agree that in a perfect world, the reasons for all design decisions would be documented.  However, in legacy software there comes a point when the only important reason that things need to work a certain way is because they have always worked that way.  (Preserving backwards compatibility is one thing about which groff users on the email list seem to nearly unanimously agree.)

No one designing software from the ground up today would have it evaluate the expression 1+2*3 as 9.  But groff must continue to do this because classical troff did this in the 1970s.

Whether the original reasons for this decision are documented is, ultimately, irrelevant: the reason this absurd precedence order persists today is so that historical troff documents continue to work as they always have.

Similarly, whether or not one understands the reasons for the various values of .hy, they have been part of the language for decades and should continue to work as they always have.

Dave <barx>
Group Member
Wed 28 Feb 2018 05:57:20 PM UTC, comment #5: 

From comment 1:

>The bug is somewhere else: The file `hyphenex.us' is read with the
>`.hpfa' request, and all data read in with this request should indeed
>be subject to `.hy'.


  No.  The "hyphenex.*" files (should) show correct hyphenations, just
as words in the ".hw" request and with the '\%' hyphenation indicator.

  Just because people have not read about the purpose or reasons for
these numbers (4 and 8) does not mean, that they do not have a reason
or a purpose.

  This lack of documenting the "why's" are bad, waste peoples time and
obscure understanding the reason for a specific code.

  So what is (was) the reason for these numbers and following that the
purpose?

N.B.
  See also bug #52457 "src: the first letter hyphenated in an English
word".

Bjarni Ingi Gislason <bjarniig>
Tue 27 Feb 2018 12:04:02 AM UTC, comment #4: 


> 3) The '.hy 8' works in 'nroff 1.22.3' (Debian 1.22.3-10)
> but not in version 1.22.3.real.441-d6ec5


Then this is a regression that needs to be addressed.

> 4) The values 4 and 8 are incompatible with the current
> method of hyphenation and make no sense in that environment.


Then the current method of hyphenation needs to be fixed to allow .hy to work as documented.  It is important for getting the best typographic results based on the column width.  If anything, groff should give the user more control over how many characters s/he wants to require before/after a hyphenation point, not less.

I will try to produce better examples.

Dave <barx>
Group Member
Mon 26 Feb 2018 11:42:37 PM UTC, comment #3: 

1) I can't reproduce the example

2) The example is too complicated

3) The '.hy 8' works in 'nroff 1.22.3' (Debian 1.22.3-10) but not in
version 1.22.3.real.441-d6ec5

4) The values 4 and 8 are incompatible with the current method of
hyphenation and make no sense in that environment.

  Groff can't hyphenate words the same way the original "troff", as it
always needs a hyphenation pattern file.

  The reasons for these two values are no longer valid, or where is it
documented?

  So these two values should be removed from the software and
documentation.

  Software can issue a warning if those are used.

  If people do not want to accept the hyphenation that "groff"
produces, the escape "\%" is there for that purpose.

  Or, if the word is not in a "hyphenex.??" file, with the ".hw"
request.

Bjarni Ingi Gislason <bjarniig>
Mon 26 Feb 2018 11:20:31 AM UTC, comment #2: 


> The file `hyphenex.us' is read with the `.hpfa'
> request, and all data read in with this request
> should indeed be subject to `.hy'.


Yes, this is the core bug.  The other cases are side issues (perhaps I should have separated them into their own bug reports, though they seemed pretty closely related to this core issue).

Regarding .hy not applying to hyphenation points defined by .hw:

> It thus makes sense to re-enable such words
> explicitly within a hyphenation exception.


If this is a design decision, then the bug is in the documentation.  The entirety of the documented behavior for ".hy 8" is "Do not hyphenate the first two characters of a word."  It makes no mention of .hw being able to overrule this.

> Again, I disagree. An explicit hyphen is always
> a place to break.


Fair enough.  This works as documented in the .cflags section.  The .hy section could perhaps be clearer about this, though arguably the scope of .hy is only hyphens added by groff, not ones in the document source.

Dave <barx>
Group Member
Sat 24 Feb 2018 05:57:39 AM UTC, comment #1: 

Some general observations.

> The user should not have to know or care whether groff
> hyphenates a word using a pattern or an exception-file
> entry; groff should follow the restrictions specified in
> .hy regardless.


I disagree.  There are languages like German where the default setup is to hyphenate both the first and last two characters (Ho-se, ei-ne, etc.).  However, it normally gives much better typographical results for long lines if you disable this.  The recommended value for both \lefthyphenmin and \righthyphenmin (to use the finer control instructions of TeX) is 3.

On the other hand, there are German prefixes like `un-' (which is kind of a negation) that are normally the best hyphenation points in the whole word.  It thus makes sense to re-enable such words explicitly within a hyphenation exception.

In other words, I don't see a bug w.r.t. to `.hw' and the `.hy' request.

The bug is somewhere else: The file `hyphenex.us' is read with the `.hpfa' request, and all data read in with this request should indeed be subject to `.hy'.

> Additionally, a user asking groff not to split words by
> leaving only two characters on a line might reasonably
> expect this to include words already containing hyphens.


Again, I disagree.  An explicit hyphen is always a place to break.  This is true for all typesetting programs I'm aware of.  If you want to suppress hyphenation after a hyphen you should use U+2011, NON-BREAKING HYPHEN, instead, or protect the word with a leading `\%' escape.

Werner LEMBERG <wl>
Group administrator
Wed 21 Feb 2018 04:55:57 AM UTC, original submission:  

For some words, groff ignores .hy settings 4 ("Do not hyphenate the last two characters of a word") and 8 ("Do not hyphenate the first two characters of a word").  This bug occurs in at least versions 1.22.2 and 1.22.3 of groff.

This example, which can be pasted directly into the shell, illustrates the bug with with ".hy 8" set:

groff -a << EOF
.ll 1i \" short enough to break the third word on each line
Visit yonder arboretum.
.br
Visit yonder archipelago.
.br
.hy 8 \"should prevent breaking after only two characters
Visit yonder arboretum.
.br
Visit yonder archipelago.
EOF

The output is:

<beginning of page>
Visit yonder ar<hy>
boretum.
Visit yonder ar<hy>
chipelago.
Visit yonder
arboretum.
Visit yonder ar<hy>
chipelago.

That is, groff follows the ".hy 8" setting for the word "arboretum" (carrying the whole word to the next line rather than breaking it) but not for "archipelago."

The bug seems to be triggered when the hyphenation for the word is explicitly provided (either by the system, in groff's hyphenation exception file, or by the user, via the .hw request) rather than inferred from a hyphenation pattern.  To verify that the exception file contains "archipelago" but not "arboretum":

egrep `echo 'arboretum|archipelago' | sed 's/./&-?/g'` /usr/share/groff/current/tmac/hyphenex.us

The user should not have to know or care whether groff hyphenates a word using a pattern or an exception-file entry; groff should follow the restrictions specified in .hy regardless.




Additionally, a user asking groff not to split words by leaving only two characters on a line might reasonably expect this to include words already containing hyphens. However, groff does not honor this restriction, either:

groff -a << EOF
.\" don't hyphenate the first and last two characters of a word
.hy 12
.
.ll 1i
He's been my go-to guy for math questions.
EOF




[Bug history: this problem was originally reported in bug #42503.  However, that bug's description was unclear and the title was misleading, and this bug tracker seems to lack the ability to update a bug's title.  So this current bug report is hopefully clearer.]

Dave <barx>
Group Member

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by gbranden (Updated the item)
  • -email is unavailable- added by bjarniig (Posted a comment)
  • -email is unavailable- added by wl (Posted a comment)
  • -email is unavailable- added by schwarze (Updated the item)
  • -email is unavailable- added by barx (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 8 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2018-06-18 gbranden StatusConfirmed Fixed
    2018-03-02 wl CategoryNone Core
        Assigned toNone wl
        Open/ClosedOpen Closed
        Planned ReleaseNone 1.22.4
    2018-03-01 wl Summary.hy values 4 and 8 ignored for some words .hy values 4 and 8 ignored for hyphenation exceptions read with .hpf or .hpfa
    2018-02-21 schwarze Severity3 - Normal 2 - Minor
        StatusNone Confirmed

    Back to the top

    Powered by Savane 3.13-3230.
    Corresponding source code