bugGNU TeXmacs - Bugs: bug #61228, Pasting UTF-8 diacritic characters...

 
 

You are not allowed to post comments on this tracker with your current authentication level.

bug #61228: Pasting UTF-8 diacritic characters from clipboard breaks their encoding when document language is localized

Submitter:  Piotr Henryk Dabrowski <dabrowski>
Submitted:  Sun 26 Sep 2021 09:22:46 PM UTC
   
 
Category:  Editor Priority:  5 - Normal
Item Group:  Error Status:  Fixed
Privacy:  Public Assigned to:  vdhoeven
Originator Name:  Open/Closed:  Closed
Release:  None Release:  2.1
Fixed Release:  None Fixed Release: 
Keywords: 

Sun 06 Nov 2022 09:33:58 PM UTC, comment #5: 

Fixed in revision 14085.

I added iso-8859-2 as one of the possible character encodings in order to make it possible to keep the old behavior when needed.

By default, we now use unicode, as suggested in the patch.

Joris van der Hoeven <vdhoeven>
Group administrator
Fri 23 Sep 2022 08:45:53 AM UTC, comment #4: 

I tried the suggestion in #1 and it works.

(file #53742)

Zoltan Kovacs <kovzol>
Sun 14 Nov 2021 08:53:54 AM UTC, comment #3: 

@Joris van der Hoeven
I tried both Auto and UTF-8 settings. The result is always the same. Also when restarting TeXmacs between testing.

Piotr Henryk Dabrowski <dabrowski>
Wed 10 Nov 2021 10:42:31 AM UTC, comment #2: 

Could you please tell us what you have in 'Edit -> Preferences -> Convert -> Verbatim -> Character encoding' ?

If you select UTF8 instead of Auto, then do things work as expected for you?

Joris van der Hoeven <vdhoeven>
Group administrator
Tue 28 Sep 2021 08:58:42 AM UTC, comment #1: 

Yes, in edit_select.cpp, selection_paste calls selection_decode, which, when the text language is Polish, Croatian, Czech, Hugarian, Slovak or Slovene, assumes the clipboard content is encoded Iso Latin 2, and performs a conversion to cork. For other languages, the text is not transcoded at that point and we get the expected characters.
 
I guess the appropriate fix is just to remove this test and conversion. And of course the fix has to be made symmetric when placing text on the clipboard.

Philippe Joyez <pjoyez>
Group Member
Sun 26 Sep 2021 09:22:46 PM UTC, original submission:  

1. Set TeXmacs document language to Polish (default with UI configured to Polish)
2. Copy those Polish diacritic characters into your clipboard from here: 'ąćęłńśźż' (ar attached txt file) 3. Paste them into the document
RESULT: Characters are pasted incorrectly (as '“Ł§ŞŤŻľË', see attached screenshot) - probably needlessly translated into Polish codepage from UTF-8

But this works perfectly fine when document language is set to Default/English! Polish diacritic characters are then pasted properly.

Typing in Polish diacritic characters from keyboard also works perfectly fine both with document language set to Polish and Default/English.

Also opening a Texmacs document with Polish diacritic characters (attached), copying them and pasting into the same document works too.
But if you try to paste them into another instance of TeXmacs, it fails in the same way as described above.

So probably TeXmacs doesn't like UTF-8 input from system clipboard, and needlessly tries to re-encode it into localized (Polish) codepage?

Environment: Kubutnu Linux 21.04, KDE, X11

Piotr Henryk Dabrowski <dabrowski>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attached Files
file #53742:  fix-61228.patch added by kovzol (1KiB - text/x-patch)
file #51970:  polish-diacritic-characters.tm added by dabrowski (168B - text/x-texmacs.doc)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by kovzol (Updated the item)
  • -email is unavailable- added by vdhoeven (Posted a comment)
  • -email is unavailable- added by pjoyez (Posted a comment)
  • -email is unavailable- added by dabrowski (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

     

    Follow 9 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2022-11-06 vdhoeven StatusConfirmed Fixed
        Assigned tofpoulain vdhoeven
        Open/ClosedOpen Closed
    2022-09-23 kovzol Attached File- Added fix-61228.patch, #53742
    2021-11-10 vdhoeven Assigned toNone fpoulain
    2021-09-28 pjoyez StatusNone Confirmed
    2021-09-26 dabrowski Attached File- Added polish-diacritic-characters.tm, #51970
        Attached File- Added polish-diacritic-characters-bug.png, #51971
        Attached File- Added polish-diacritic-characters.txt, #51972

    Back to the top

    Powered by Savane 3.13-d3ae.
    Corresponding source code