bugGNU roff - Bugs: bug #48904, Hyphenation patterns of type...

 
 

bug #48904: Hyphenation patterns of type '.a[135]' do not work

Submitter:  Bjarni Ingi Gislason <bjarniig>
Submitted:  Thu 25 Aug 2016 07:24:39 PM UTC
   
 
Category:  Core Severity:  3 - Normal
Item Group:  Incorrect behaviour Status:  Fixed
Privacy:  Public Assigned to:  wl
Open/Closed:  Closed Planned Release:  1.22.4
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Sat 22 Jan 2022 12:44:56 PM UTC, comment #7: 


commit b5aaab51158255411fd196b43f70f45dda511a42
Author: Werner Lemberg <wl@gnu.org>
Date:   Thu Mar 1 20:17:21 2018 +0100

    Fix some problems related to hyphenation.

diff --git a/ChangeLog b/ChangeLog
index 1506e57dd..9ce1d1d04 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,26 @@
+2018-03-01  Werner LEMBERG  <wl@gnu.org>
+
+       Fix some problems related to hyphenation.
+
+       Introduce values 16 (allow hyphenation before last character) and 32
+       (allow hyphenation before first character) to the `hy' request
+       (#48904).
+
+       Really fix problem reported in issue #48904.
+
+       Fix #53243, which is a regression introduced in 2016-09-09.
+
+       * src/roff/troff/env.cpp (HYPHEN_LAST_CHAR, HYPHEN_FIRST_CHAR): New
+       enumeration values.
+       (hyphenate_request): Check for contradicting values.
+       (environment::hyphenate_line): Fix minimum word length considered
+       for hyphenation.
+       (environment::print_env): Updated.
+       (hyphenate): Fix and extend logic.  The original patch to handle
+       #48904 was bad.
+
+       * doc/groff.texi, man/groff_diff.7.man, NEWS: Updated.


G. Branden Robinson <gbranden>
Group administrator
Thu 01 Mar 2018 07:31:30 PM UTC, comment #6: 

The original fix for this issue was flawed; it is now fixed with commit b5aaab51.  I've also implemented `.hy' values 16 and 32, as suggested.

Werner LEMBERG <wl>
Group administrator
Tue 20 Sep 2016 05:09:37 AM UTC, comment #5: 

Regarding wrong hyphenation patterns: Please contact its author(s)!  groff simply takes what is available, without changes in the patterns.  Note that it usually doesn't make any sense to comment on single patterns if they are derived with patgen (as is the case for `hyphen.sv') – there exists at least a single word in the original set of hyphenated words that uses it.

http://tug.org/tex-hyphen/

The patterns for US-English were also derived from a set of words, ignoring the first two and the last three characters of a word (you have to specify these two values while generating the patterns).  This is a bit unfortunate since groff only can suppress two characters at the end, not three – it can thus happen that some words are incorrectly hyphenated because Liang's algorithm might produce artificial breakpoints while applying the patterns.  In particular, you must set .hy to 12 or 14 for the US-English patterns, or you get much worse results.

Werner LEMBERG <wl>
Group administrator
Sun 18 Sep 2016 11:57:56 PM UTC, comment #4: 

  Thanks for the fix.  The hyphenation is now as expected.

  There is a least one pattern that is wrong (outdated), '.i3so' in the
"hyphen.sv" file.  The 'i' there is not a prefix.

  By using ".hy 4" some hyphenation pattern are never tested.

  Programs should not hide any kind of (input) errors.  (Their authors and
maintainers are using "All errors (warnings) are equal but some are more
equal than others.)

  Some comparisons in the "hyphen.us"

.hy hyphenation never
--- ----------- ----
1 hy\%phen\%a\%tion n\%ev\%er (error, from 'n5eve)
12|14 hyphen\%ation never


  It looks like that some hyphenation points are derived from syllables
rather than genuine hyphenation, that takes pronunciation into account.


Bjarni Ingi Gislason <bjarniig>
Fri 09 Sep 2016 06:50:19 AM UTC, comment #3: 

`.hy 8' is wrong for the Swedish patterns, it must be `.hy 4'; and the `.hy' flags were incorrectly handled by troff.

Both is now fixed in git, thanks for the report.  Please test.

Werner LEMBERG <wl>
Group administrator
Sun 28 Aug 2016 11:02:09 PM UTC, comment #2: 


  No, because

a) the file "sv.tmac" contains the '.hy 8' request

b) even if it did not, there is enough space to break the word 'ostra' at
'ost-ra' in the example.  If the line length is shortened to 19n there is no
breakage.
  To achieve that a '.hw o-st-ra' request is needed.

c) and evidently a one-letter prefix is never hyphenated (except in a 'hw' request), even with a valid pattern.

  Case 'c' is caused by a hard-coded case.  It is so in TEX (at least in the
original) and probably in the original "roff", as it is constructed for
English.

  So the restriction 'no hyphenation of a one-letter prefix' should be
dropped (if a file with hyphenation patterns is used).

  Add a number to the 'hy' request to prohibit a hyphenation of a one-letter
prefix.

  The "Heirloom-troff" has added the values 16 and 32 to allow a
hyphenation of a one-letter suffix and prefix respectively, but that only
has an effect for explicit hyphenation marks according to the documentation
dated 17th September 2007.  These numbers should work with hyphenation
patterns and actually prohibit, not allow, some specific hyphenation, just
as the traditional numbers greater than one do.

N.B.

  The value 8 for '.hy' in 'sv.tmac' is wrong as there exist hyphenation
patterns of the form '.xy[135]' in Swedish.

  There is also no explanation why a certain value is selected.

  It is not the right of the author of a global tmac-file to include a
specific value (his own preference) for 'hy' except 0 with an explanation.
Only the user should change that for himself, if he wants reduced
hyphenations.


Bjarni Ingi Gislason <bjarniig>
Fri 26 Aug 2016 04:46:55 PM UTC, comment #1: 

No time for a detailed analysis.  However, if I say


groff -msv -Tlatin1 sv.tr


using


.ll 20n
Sama ostra
Sama ostra
Sama ostra
Sama ostra
Sama ostra
Sama ostra
Sama ostra
Sama ostra
Sama ostra
Sama ostra
Sama ostra
Sama ostra
Sama ostra
Sama ostra
Sama ostra
Sama ostra


as input, I get


Sama ostra Sama ost-
ra Sama  ostra  Sama
ostra   Sama   ostra
Sama ostra Sama ost-
ra  Sama  ostra Sama
ostra   Sama   ostra
Sama ostra Sama ost-
ra Sama  ostra  Sama
ostra   Sama   ostra
Sama ostra


which looks correct.  Do you expect a different output?

Werner LEMBERG <wl>
Group administrator
Thu 25 Aug 2016 07:24:39 PM UTC, original submission:  

GROFF VERSION:
1.22.3

MACHINE:
GA-E350N Win8 from Gigabyte

OS:
Linux ... 3.16.36-1 #1 SMP Sun Jul 10 18:52:03 UTC 2016 x86_64 GNU/Linux

COMPILER:
That used by Debian groff (1.22.3-8)

INPUT FILES:
.fam C
.\" Width of each character is 6 points, 1m is 10 points
.ll 10n \" 50 points or 8.3 letters
.hla sv
.hpf hyphen.sv
.na
.hy 1
.\" hyphen.sv contains the pattern '.o3stra'
Sama ostra

COMMAND LINE:
groff -b -ww -Z <input file>

DESCRIPTION OF INCORRECT BEHAVIOUR:
  Words are not hyphenated according to a pattern of type '.a[135]b'.

  Output:

x T ps
x res 72000 1 1
x init
p1
x font 6 CR
f6
s10000
V12000
H72000
md
DFd
tSama
n12000 0
V24000
H72000
tostra
n12000 0
x trailer
V792000
x stop

Bjarni Ingi Gislason <bjarniig>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by gbranden (Posted a comment)
  • -email is unavailable- added by wl (Posted a comment)
  • -email is unavailable- added by bjarniig (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 7 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2022-01-22 gbranden CategoryNone Core
        Item GroupNone Incorrect behaviour
    2016-09-09 wl StatusNeed Info Fixed
        Assigned toNone wl
        Open/ClosedOpen Closed
        Planned ReleaseNone 1.22.4
    2016-08-26 wl StatusNone Need Info

    Back to the top

    Powered by Savane 3.13-758e.
    Corresponding source code