bugGNU Screen - Bugs: bug #50044, Update lookup tables to support...

 
 

bug #50044: Update lookup tables to support Unicode 9.0.0

Submitted by:  None
Submitted on:  Thu 12 Jan 2017 10:53:36 PM UTC  
Votes:  100  
 
Category: Feature RequestSeverity: 3 - Normal
Priority: 5 - NormalStatus: Fixed
Privacy: PublicAssigned to: Amadeusz Sławiński <amade>
Open/Closed: ClosedRelease: 4.99.0
Fixed Release: 4.6.0Planned Release: 4.6.0
Work Required: None

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

(Jump to the original submission Jump to the original submission)

Thu 22 Jun 2017 09:06:44 AM UTC, comment #6:

Thanks for patch. Applied it to both screen-v4 and master.

I did some browsing with elinks and w3m on random translated wikipedia pages and seemed to work mostly fine, although it also depends on unicode support in those applications.

Amadeusz Sławiński <amade>
Project AdministratorIn charge of this item.
Wed 21 Jun 2017 08:13:14 PM UTC, comment #5:

I also use weechat inside screen, and this is now affecting me quite badly. Applying the attached patch to screen fixes it for me. I think the problem is now worse with Debian Stretch having been released, as weechat compiled with a newer glibc and gnome-terminal compiled with a newer glib are now out-of-sync with screen, leading to severe display corruption problems.

David Taylor <davidt84>
Tue 07 Feb 2017 06:05:10 PM UTC, comment #4:

Attaching a patch against screen-v4 of my second patch that passes my rudimentary tests.

(file #39691)

Brian De Wolf <bldewolf>
Tue 07 Feb 2017 01:01:55 AM UTC, comment #3:

I am attaching a new patch against master that also updates the ambiguous table. I have left the formatting as one per line as otherwise the diffs can easily become unreadable.

This patch works in my brief testing but it really should be tested by a heavy wide character user, e.g. someone working with East Asian characters.

(file #39686)

Brian De Wolf <bldewolf>
Thu 12 Jan 2017 10:59:34 PM UTC, comment #2:

Oh, it doesn't like the emoji. Third time's a charm:

I use weechat inside a screen session inside mate-terminal for IRC and lately emoji have begun showing up and my terminal gets corrupted. I was able to track this down to screen's wide character handling. Because screen doesn't correctly detect that new emoji (such as U+1F525) are wide characters, it inserts cursor-forward commands where none are necessary.

For example, weechat will print an emoji at 20,20 and 22,20. Screen will receive this and print the character at 20,20, emit cursor-forward (^[C), then print the next emoji at 22,20. This causes mate-terminal to see what is actually a 2 character sequence as a 3 character sequence. This causes all sorts of havoc as later character movements are distorted, leaving emoji and other characters unerased on the screen.

Anyway, I tracked this down to glib (what mate-terminal uses to detect widths of characters) being Unicode 9.0.0 compatible. To test my theory, I patched in the tables that glib uses into screen. With this patch, the newer wide characters appear and are manipulated correctly. It replaces a smaller set of tests which appear to be a subset of the ranges that are now checked.

The table is taken from https://raw.githubusercontent.com/GNOME/glib/glib-2-50/glib/gunichartables.h and is a product of https://github.com/GNOME/glib/blob/glib-2-50/glib/gen-unicode-tables.pl

Note: My patch does not update the ambiguous tables, which might also need updating.

Brian De Wolf <bldewolf>
Thu 12 Jan 2017 10:57:33 PM UTC, comment #1:

I didn't have an account, so it cut off my post. Here's the whole thing again:

I use weechat inside a screen session inside mate-terminal for IRC and lately emoji have begun showing up and my terminal gets corrupted. I was able to track this down to screen's wide character handling. Because screen doesn't correctly detect that new emoji (such as

Brian De Wolf <bldewolf>
Thu 12 Jan 2017 10:53:36 PM UTC, original submission:

I use weechat inside a screen session inside mate-terminal for IRC and lately emoji have begun showing up and my terminal gets corrupted. I was able to track this down to screen's wide character handling. Because screen doesn't correctly detect that new emoji (such as

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

Attached Files
file #39691:  screen-update-unicode-wide-tables-screen-v4.patch added by bldewolf (11KiB - text/x-patch - same patch as -master but against screen-v4 and using its tabbing style)
file #39686:  screen-update-unicode-wide-tables-master.patch added by bldewolf (10KiB - text/x-patch - new version against master that updates the ambiguous table as well)
file #39454:  screen-update-unicode-wide-tables.patch added by None (3KiB - text/x-patch)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by amade (Posted a comment)
  • -unavailable- added by davidt84 (Posted a comment)
  • -unavailable- added by davidt84 (Voted in favor of this item)
  • -unavailable- added by bldewolf (Posted a comment)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 100 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 9 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Thu 29 Jun 2017 10:15:21 AM UTCamadeOpen/ClosedOpen=>Closed
      Fixed ReleaseNone=>4.6.0
    Thu 22 Jun 2017 09:06:44 AM UTCamadeStatusNone=>Fixed
      Assigned toNone=>amade
      Planned ReleaseNone=>4.6.0
    Wed 21 Jun 2017 08:08:22 PM UTCdavidt84Carbon-Copy-=>Added davidt84
    Tue 07 Feb 2017 06:05:10 PM UTCbldewolfAttached File-=>Added screen-update-unicode-wide-tables-screen-v4.patch, #39691
    Tue 07 Feb 2017 01:01:55 AM UTCbldewolfAttached File-=>Added screen-update-unicode-wide-tables-master.patch, #39686
    Thu 12 Jan 2017 10:53:36 PM UTCNoneAttached File-=>Added screen-update-unicode-wide-tables.patch, #39454

    Back to the top


    Powered by Savane 3.1-cleanup1