Wed 24 May 2006 12:20:51 AM UTC, original submission:
BACKGROUND
According to Unicode Standard Annex #11(UAX #11) [EAW], some characters
are classified as East Asian Ambiguous (A) Class. Class A characters can
be sometimes wide (1 Em wide) and sometimes narrow (1/2 Em wide).
Quotes from UAX#11, Section 4:
Ambiguous characters occur in East Asian legacy character sets as wide
characters, but as narrow (i.e. normal-width) characters in non-East
Asian usage (Examples are the Greek and Cyrillic alphabet found in East
Asian character sets, but also some of the mathematical symbols).
That is, legacy East Asian softwares display Class A characters as wide
as CJK ideograph, but non-East Asian softwares display them as wide as
normal Latin characters.
PROBLEM DESCRIPTION
Since screen is not aware of display end's locale, it assumes the display
end is non-East Asian one. Such assumption leads to a result of chaos in
East Asian usage. For example, screen is translating a character 0xA1B4(in
Big5 encoding) from a window to a Unicode(UTF-8) display. 0xA1B4 in Big5
is "wide"(occupied 2 Lating characters' width), and mapped to U+25CF in
Unicode, which is an East Asian Ambiguous character. Because of the
assumption described above, screen will "pad" a space following U+25CF to
justify the width. This is good if the display end is a non-East Asian
terminal, but not good at all in an East Asian terminal.
IMPACT
Because East Asian terminals display Class A characters as wide characters,
the extra "padding"(spaces) will mess up the output.
FIX
A reasonable solution is adding a global(display) option, for example,
named "cjkwidth". When cjkwidth is on, the encoding.c:utf8_isdouble() will
return true for Class A characters. That is, screen will not pad extra
spaces when cjkwidth is on.
The following patch adds the cjkwidth option(default: off) and turns it on
automatically in an East Asian locale(zh_, ja_, ko_*). The modification
of encoding.c:utf8_isdouble() is based on Markus Kuhn's free wcwidth()
implementation [MGK].
REFERENCES:
[EAW] Unicode Standard Annes #11: East Asian Width
http://www.unicode.org/reports/tr11/tr11-11.html
[MGK] Markus Kuhn's free wcwidth() implementation
http://www.cl.cam.ac.uk/~mgk25/ucs/wcwidth.c
ACKNOWLEDGMENTS:
Kuang-che Wu <kcwu AT csie.org> provided an early version of
utf8_isdouble() implementation and useful information for creating this
patch. Rong-En Fan <rafan AT infor.org> also provided some comments and
pushed me to complete this problem report.
Yi-Hsuan Hsin
<mhsin AT mhsin.org>
|