3 |
java.lang.Character allows one to retrieve information on all 38,887 |
java.lang.Character allows one to retrieve information on all 38,887 |
4 |
characters of the Unicode character set. This is a lot of data. The |
characters of the Unicode character set. This is a lot of data. The |
5 |
database specification outlined here is meant to be fast, small, and |
database specification outlined here is meant to be fast, small, and |
6 |
upgradable to new versions of the Unicode 2 specification (minus |
upgradable to new versions of the Unicode 3 specification (minus |
7 |
Character.Subset information) by running a script on the data files |
Character.UnicodeBlock information) by running a script on the data |
8 |
that the Unicode Consortium distributes. |
files that the Unicode Consortium distributes. |
9 |
|
|
10 |
The database consists of three files: |
The database consists of three files: |
11 |
1) character.uni (main database of character attributes) |
1) character.uni (main database of character attributes) |
14 |
|
|
15 |
File sizes for Unicode 2.1.2 spec |
File sizes for Unicode 2.1.2 spec |
16 |
--------------------------------- |
--------------------------------- |
17 |
character.uni: 16359 bytes |
character.uni: 8760 bytes |
18 |
block.uni : 4995 bytes |
block.uni : 4122 bytes |
19 |
titlecase.uni: 16 bytes |
titlecase.uni: 72 bytes |
20 |
|
|
21 |
|
File sizes for Unicode 3.0.0 spec |
22 |
|
--------------------------------- |
23 |
|
character.uni: 9776 bytes |
24 |
|
block.uni : 4938 bytes |
25 |
|
titlecase.uni: 48 bytes |
26 |
|
|
27 |
|
File sizes for Unicode 3.2.0 spec |
28 |
|
--------------------------------- |
29 |
|
character.uni: 10784 bytes |
30 |
|
block.uni : 5586 bytes |
31 |
|
titlecase.uni: 48 bytes |
32 |
|
|
33 |
All quantities are unsigned unless otherwise specified. |
All quantities are unsigned unless otherwise specified. |
34 |
All quantities are stored in big endian format. |
All quantities are stored in big endian format. |
35 |
|
|
36 |
character.uni |
character.uni |
37 |
------------- |
------------- |
38 |
Some characters in the Unicode specification have an entry in the |
All characters in the Unicode specification have an attributes entry |
39 |
character.uni file (characters in compressed blocks do not). Characters |
in the character.uni file (alhough compression is used to share |
40 |
are stored sequentially, based on the Unicode character number, and |
entries where possible). Characters are stored sequentially, based on |
41 |
there are no null entries. Each entry consists of 7 bytes. |
the Unicode character number within a block, although multiple blocks |
42 |
|
may point to the same character attribute location. There are no null |
43 |
C = Category |
entries. Each entry consists of 8 bytes. The file is limited to 4096 |
44 |
N = Numerical Decimal Value |
entries (32768 bytes), without making changes to the semantics of |
45 |
(0xFFFF if unused, 0xFFFE if not representable as nonnegative |
block.uni. |
46 |
integer value. If category is "Nd", then this is also the |
|
47 |
decimal digit value.) |
C = Category (values correspond to constants in Character. For |
48 |
S = No-break version of a space (1/0) |
example, "Nd" maps to 9.) |
49 |
U = Uppercase mapping (0 = no mapping) |
N = Numerical Decimal Value: signed. -1 if unused, -2 if not |
50 |
L = Lowercase mapping (0 = no mapping) |
representable as nonnegative integer value, otherwise a |
51 |
|
non-negative value. (For example, this is the decimal digit |
52 |
|
value of the category "Nd". Note that Java also attaches a value |
53 |
|
to some sequences of 'a'-'z'; see Character.getNumericValue(char) |
54 |
|
for details.) In a compressed block, subsequent characters must |
55 |
|
adjust their value by their distance from the start of the block |
56 |
|
if the value is non-negative. |
57 |
|
S = Non-breaking space (1/0) |
58 |
|
U = Offset to uppercase mapping: signed. Characters with no mapping |
59 |
|
will have 0, which maps back to itself. Otherwise, the value of |
60 |
|
toUpperCase(ch) is (char) (ch - U). |
61 |
|
L = Offset to lowercase mapping: signed. Characters with no mapping |
62 |
|
will have 0, which maps back to itself. Otherwise, the value of |
63 |
|
toLowerCase(ch) is (char) (ch - L). |
64 |
|
D = Directional category: signed (Values correspond to constants in |
65 |
|
Character, with -1 meaning unassigned. For example, 'a' has |
66 |
|
direction "L", which maps to 0.) |
67 |
|
M = Mirrored (1/0) |
68 |
x = Empty |
x = Empty |
69 |
|
|
70 |
xxSCCCCC NNNNNNNN NNNNNNNN UUUUUUUU UUUUUUUU |
xMSCCCCC NNNNNNNN NNNNNNNN UUUUUUUU UUUUUUUU |
71 |
\________/ \________/ \________/ \________/ \________/ |
\________/ \________/ \________/ \________/ \________/ |
72 |
byte 6 byte 5 byte 4 byte 3 byte 2 |
byte 7 byte 6 byte 5 byte 4 byte 3 |
73 |
|
|
74 |
LLLLLLLL LLLLLLLL |
LLLLLLLL LLLLLLLL DDDDDDDD |
75 |
\________/ \________/ |
\________/ \________/ \________/ |
76 |
byte 1 byte 0 |
byte 2 byte 1 byte 0 |
77 |
|
|
78 |
ranges of values in Unicode 2.1.2 spec |
ranges of values |
79 |
|
|
80 |
C = 0..28 (Sun uses 0..28, and skips 17, so that's what we do too) |
C = 0..30 (Sun skips 17, so that's what we do too) |
81 |
B = 1..69 |
N = -2..10000 |
82 |
N = 0..10000 |
D = -1..18 |
|
D = 0..9 |
|
83 |
|
|
84 |
block.uni |
block.uni |
85 |
--------- |
--------- |
86 |
Characters within the Unicode specification tend to come in blocks -- |
Characters within the Unicode specification tend to come in blocks -- |
87 |
sets of sequential characters. The Classpath Unicode database takes |
sets of sequential characters. The Classpath Unicode database takes |
88 |
advantage of this property. Each entry in the block.uni file consists |
advantage of this property. Each entry in the block.uni file consists |
89 |
of 9 bytes. Entries are stored sequentially, based on the Unicode |
of 6 bytes. Entries are stored sequentially, based on the Unicode |
90 |
character number which starts a block. If the compressed bit is set, |
character number which starts a block. If the compressed bit is set, |
91 |
then there is only one entry for this block in the character.uni file. |
then offset O is the only entry for this block in the character.uni |
92 |
That entry in the character.uni file represents the attributes of all |
file, and represents the attributes of all characters in the |
93 |
the characters of that block. |
block. Otherwise, the characters in the block occupy sequential |
94 |
|
attributes, starting with character S at offset O. |
|
Note: For Unicode 2.1.2, compressed blocks are mandatory for: |
|
|
|
|
|
U+4E00 - U+9FFF: The CJK Ideographs Area |
|
|
U+AC00 - U+D7A3: The Hangul Syllables Area |
|
|
U+D800 - U+DFFF: The Surrogates Area |
|
|
U+E000 - U+F8FF: The Private Use Area |
|
|
U+F900 - U+FAFF: CJK Compatibility Ideographs |
|
95 |
|
|
96 |
S = Unicode character which represents start of block |
S = Unicode character which represents start of block |
97 |
E = Unicode character which represents end of block |
E = Unicode character which represents end of block |
98 |
O = Offset of this block within the character.uni file |
O = Offset of this block within the character.uni file |
99 |
C = Compressed |
C = Compressed (1/0) If compressed, all characters in the block share |
100 |
x = Empty |
a common attribute, otherwise the characters in the block occupy |
101 |
|
sequential attributes starting at offset. |
102 |
SSSSSSSS SSSSSSSS EEEEEEEE EEEEEEEE xxxxxxxC |
|
103 |
\________/ \________/ \________/ \________/ \________/ |
SSSSSSSS SSSSSSSS EEEEEEEE EEEEEEEE |
104 |
byte 8 byte 7 byte 6 byte 5 byte 4 |
\________/ \________/ \________/ \________/ |
105 |
|
byte 5 byte 4 byte 3 byte 2 |
106 |
|
|
107 |
OOOOOOOO OOOOOOOO OOOOOOOO OOOOOOOO |
COOOOOOO OOOOOOOO |
108 |
\________/ \________/ \________/ \________/ |
\________/ \________/ |
109 |
byte 3 byte 2 byte 1 byte 0 |
byte 1 byte 0 |
110 |
|
|
111 |
titlecase.uni |
titlecase.uni |
112 |
------------- |
------------- |
113 |
Characters in which the titlecase differs from the uppercase are |
Characters in which the titlecase differs from the uppercase are |
114 |
stored in titlecase.uni. There are only four characters in the |
stored in titlecase.uni. There are only twelve characters in the |
115 |
Unicode 2.1.2 specification which fit this description, and it's |
Unicode 3.0.0 specification which fit this description, and it's |
116 |
doubtful that any others will ever be added to the specification. |
doubtful that any others will ever be added to the specification. |
117 |
However, we should be able to support more, without changing |
However, we should be able to support more, without changing |
118 |
java.lang.Character, and this is why we have not hardcoded these |
java.lang.Character, and this is why we have not hardcoded these |