bugGNU Screen - Bugs: bug #16666, [PATCH] Support for UAX#11 East...

 
 

bug #16666: [PATCH] Support for UAX#11 East Asian Ambiguous characters

Submitted by:  None
Submitted on:  Wed 24 May 2006 12:20:51 AM UTC  
Votes:  11  
 
Category: Program LogicSeverity: 3 - Normal
Priority: 5 - NormalStatus: Fixed
Privacy: PublicAssigned to: None
Open/Closed: ClosedRelease: None
Fixed Release: 4.1.0Planned Release: 4.1.0
Work Required: 0 - Hours

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

Thu 07 Aug 2008 09:40:51 AM UTC, comment #1:

Fixed for 4.1.0 in 6f4b089

Micah Cowan <micahcowan>
Project Member
 Spam posted by an anonymous
 Spam posted by an anonymous
 Spam posted by an anonymous
 Spam posted by an anonymous
 Spam posted by an anonymous
Wed 24 May 2006 12:20:51 AM UTC, original submission:

BACKGROUND

According to Unicode Standard Annex #11(UAX #11) [EAW], some characters
are classified as East Asian Ambiguous (A) Class. Class A characters can
be sometimes wide (1 Em wide) and sometimes narrow (1/2 Em wide).

Quotes from UAX#11, Section 4:
Ambiguous characters occur in East Asian legacy character sets as wide
characters, but as narrow (i.e. normal-width) characters in non-East
Asian usage (Examples are the Greek and Cyrillic alphabet found in East
Asian character sets, but also some of the mathematical symbols).

That is, legacy East Asian softwares display Class A characters as wide
as CJK ideograph, but non-East Asian softwares display them as wide as
normal Latin characters.

PROBLEM DESCRIPTION

Since screen is not aware of display end's locale, it assumes the display
end is non-East Asian one. Such assumption leads to a result of chaos in
East Asian usage. For example, screen is translating a character 0xA1B4(in
Big5 encoding) from a window to a Unicode(UTF-8) display. 0xA1B4 in Big5
is "wide"(occupied 2 Lating characters' width), and mapped to U+25CF in
Unicode, which is an East Asian Ambiguous character. Because of the
assumption described above, screen will "pad" a space following U+25CF to
justify the width. This is good if the display end is a non-East Asian
terminal, but not good at all in an East Asian terminal.

IMPACT

Because East Asian terminals display Class A characters as wide characters,
the extra "padding"(spaces) will mess up the output.

FIX

A reasonable solution is adding a global(display) option, for example,
named "cjkwidth". When cjkwidth is on, the encoding.c:utf8_isdouble() will
return true for Class A characters. That is, screen will not pad extra
spaces when cjkwidth is on.

The following patch adds the cjkwidth option(default: off) and turns it on
automatically in an East Asian locale(zh_, ja_, ko_*). The modification
of encoding.c:utf8_isdouble() is based on Markus Kuhn's free wcwidth()
implementation [MGK].

REFERENCES:

[EAW] Unicode Standard Annes #11: East Asian Width
http://www.unicode.org/reports/tr11/tr11-11.html

[MGK] Markus Kuhn's free wcwidth() implementation
http://www.cl.cam.ac.uk/~mgk25/ucs/wcwidth.c

ACKNOWLEDGMENTS:

Kuang-che Wu <kcwu AT csie.org> provided an early version of
utf8_isdouble() implementation and useful information for creating this
patch. Rong-En Fan <rafan AT infor.org> also provided some comments and
pushed me to complete this problem report.

Yi-Hsuan Hsin
<mhsin AT mhsin.org>

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

Attached Files
file #10039:  patch-cjkwidth-cvs-2006052001 added by None (9KiB - application/octet-stream - patch-cjkwidth)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by micahcowan (Updated the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 11 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 10 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Thu 07 Aug 2008 09:40:51 AM UTCmicahcowanStatusNeed Investigation=>Fixed
      Open/ClosedOpen=>Closed
      Fixed ReleaseNone=>4.1.0
    Mon 23 Jun 2008 09:49:09 PM UTCmicahcowanCategoryNone=>Program Logic
      Planned ReleaseNone=>4.1.0
      Work RequiredNone=>0 - Hours
    Sun 15 Jun 2008 06:36:25 AM UTCmicahcowanStatusNone=>Need Investigation
    Thu 19 Apr 2007 08:34:03 AM UTCnoxCarbon-Copy-=>Added nox
    Mon 08 Jan 2007 08:23:55 AM UTCjsliCarbon-Copy-=>Added jsli
    Wed 24 May 2006 12:20:51 AM UTCNoneAttached File-=>Added patch-cjkwidth-cvs-2006052001, #10039

    Back to the top


    Powered by Savane 3.1-cleanup1