bugGNU TeXmacs - Bugs: bug #60284, Hyphenation doesn’t work with...

 
 

You are not allowed to post comments on this tracker with your current authentication level.

bug #60284: Hyphenation doesn’t work with Russian language (only)

Submitted by:  Vladimir <vcheese>
Submitted on:  Wed 24 Mar 2021 12:17:45 PM UTC  
 
Category:  Editor Priority:  5 - Normal
Item Group:  Error Status:  None
Privacy:  Public Assigned to:  Joris van der Hoeven <vdhoeven>
Originator Name:  Open/Closed:  Open
Release:  1.99.19 Release:  None
Fixed Release:  None Fixed Release: 
Keywords: 

Thu 08 Apr 2021 09:49:56 PM UTC, comment #4: 

It seems that the problem lies in the "shorter_box_rep" class in "src/Typeset/Boxes/Modifier/modifier_boxes.cpp", which is the type of box used for the first part of a hyphenated word.

The variable "len" in that class is the length of the string in the shorter box, measured in utf8 characters, e.g. len=4 for "пере". It is, however, used in that class to increment paths, which increase in steps of 6 per utf8 character, as they are represented as "<#443>", for example.

JW <jeronim>
Thu 08 Apr 2021 01:09:00 PM UTC, comment #3: 

Thanks for including the patch. After some further testing it seems that unfortunately there is an additional problem.

With the patch the cursor cannot move properly through hyphenated words. The cursor jumps around to random positions and on the terminal I get messages like

warning: invalid cursor position [ 1, 1, 166 ]

From the output of "Debug->Status->Path" it seems that the path normally increases by 6 when moving though Cyrillic characters, as may be expect from the internal representation of utf8. When moving through a hyphenated word, jumps of various lengths occurs, not just 6.

I have tried to find out where this happens, but have been unable to find the root of the problem. Until such time, it may be better to revert the patch.

JW <jeronim>
Wed 24 Mar 2021 11:10:28 PM UTC, comment #2: 

I believe the attached patch fixes the problem. Lowercasing is also enabled for the exception case as the exception lists. I can't find a reference for the casing in the exception list, though. It would be good to verify what the convention for German is as words are written with a starting uppercase letter.

(file #51134)

JW <jeronim>
Wed 24 Mar 2021 10:15:51 PM UTC, comment #1: 

This bug seems to be due to the use of the "locase_all" function in "src/System/Language/hyphenate.cpp"

Changing the line

s= "." * locase_all (s) * ".";

to

s= "." * s * ".";

results in hyphenation of lowercase words. It seems "locase_all" can't handle Cyrillic characters.

JW <jeronim>
Wed 24 Mar 2021 12:17:45 PM UTC, original submission:  

I done everything which in Help → Notes for users of Cyrillic languages:
  * selected Russian as default language in Edit → Preferences → Language → Russian;
  * selected Russian for an entire document using Document → Language → Russian;
  * I also checked if a file with patterns exists on the path /usr/share/TeXmacs/langs/natural/hyphen/hyphen.russian;
  * I tried to force hyphenation by Format → Hyphenate as. But nothing has changed.

Despite this hyphenation doesn’t work with Russian language. But if I set any other language (I tried English and German), then everything works correctly.

At the same time, the spell check of the Russian language works.

Is it a bug? How to fix this issue?

An example file is attached to the message.

GNU/Linux
5.10.23-1-MANJARO
TeXmacs version 1.99.19

Vladimir <vcheese>

 

Attached Files
file #51134:  hyphen-russian.diff added by jeronim (968B - text/x-patch)
file #51126:  test.tm added by vcheese (8KiB - text/x-texmacs.doc)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by mgubi (Updated the item)
  • -email is unavailable- added by mgubi
  • -email is unavailable- added by jeronim (Posted a comment)
  • -email is unavailable- added by vcheese (Submitted the item)
  • -email is unavailable- added by vcheese
  •  

    Do you think this task is very important?
    If so, you can add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

     

     

    Follow 6 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2021-04-08 mgubi Assigned toNone => vdhoeven
        Carbon-Copy- => Added jo
    2021-03-24 jeronim Attached File- => Added hyphen-russian.diff, #51134
    2021-03-24 vcheese Attached File- => Added test.tm, #51126
        Attached File- => Added Screenshot_20210324_183615.png, #51127
        Carbon-Copy- => Added vcheese@ya.ru

    Back to the top


    Powered by Savane 3.6