Fixed for 4.1.0 in 6f4b089

BACKGROUND

  According to Unicode Standard Annex #11(UAX #11) [EAW], some characters
  are classified as East Asian Ambiguous (A) Class. Class A characters can
  be sometimes wide (1 Em wide) and sometimes narrow (1/2 Em wide).

  Quotes from UAX#11, Section 4:
    Ambiguous characters occur in East Asian legacy character sets as wide
    characters, but as narrow (i.e. normal-width) characters in non-East
    Asian usage (Examples are the Greek and Cyrillic alphabet found in East
    Asian character sets, but also some of the mathematical symbols).

  That is, legacy East Asian softwares display Class A characters as wide
  as CJK ideograph, but non-East Asian softwares display them as wide as
  normal Latin characters.

PROBLEM DESCRIPTION

  Since screen is not aware of display end's locale, it assumes the display
  end is non-East Asian one. Such assumption leads to a result of chaos in
  East Asian usage. For example, screen is translating a character 0xA1B4(in
  Big5 encoding) from a window to a Unicode(UTF-8) display. 0xA1B4 in Big5
  is "wide"(occupied 2 Lating characters' width), and mapped to U+25CF in
  Unicode, which is an East Asian Ambiguous character. Because of the
  assumption described above, screen will "pad" a space following U+25CF to
  justify the width. This is good if the display end is a non-East Asian
  terminal, but not good at all in an East Asian terminal.

IMPACT

  Because East Asian terminals display Class A characters as wide characters,
  the extra "padding"(spaces) will mess up the output.

FIX

  A reasonable solution is adding a global(display) option, for example,
  named "cjkwidth". When cjkwidth is on, the encoding.c:utf8_isdouble() will
  return true for Class A characters. That is, screen will not pad extra
  spaces when cjkwidth is on.

  The following patch adds the cjkwidth option(default: off) and turns it on
  automatically in an East Asian locale(zh_*, ja_*, ko_*). The modification
  of encoding.c:utf8_isdouble() is based on Markus Kuhn's free wcwidth()
  implementation [MGK].


REFERENCES:

  [EAW]    Unicode Standard Annes #11: East Asian Width
           http://www.unicode.org/reports/tr11/tr11-11.html

  [MGK]    Markus Kuhn's free wcwidth() implementation
           http://www.cl.cam.ac.uk/~mgk25/ucs/wcwidth.c

ACKNOWLEDGMENTS:

  Kuang-che Wu <kcwu AT csie.org> provided an early version of
  utf8_isdouble() implementation and useful information for creating this
  patch. Rong-En Fan <rafan AT infor.org> also provided some comments and
  pushed me to complete this problem report.



Yi-Hsuan Hsin
<mhsin AT mhsin.org>