bugGNU Octave - Bugs: bug #63139, About string support for Chinese,...

 
 

bug #63139: About string support for Chinese, Japanese and Korean characters

Submitter:  None
Submitted:  Fri 30 Sep 2022 12:28:08 PM UTC
   
 
Category:  Other Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Matlab Compatibility
Status:  Duplicate Assigned to:  None
Originator Name:  Han Originator Email:  -email is unavailable-
Open/Closed:  * Closed Release:  * 7.2.0
Operating System:  * GNU/Linux Fixed Release:  None
Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Fri 30 Sep 2022 01:51:33 PM UTC, comment #3: 

Maybe more relevant to the topic here: bug #49348.

Re-reading that bug, it looks like this is a duplicate to that older report.

Closing as duplicate.

Markus Mützel <mmuetzel>
Group administrator
Fri 30 Sep 2022 01:21:02 PM UTC, comment #2: 

See also this related thread about umlauts: bug #47571

Arun Giridhar <arungiridhar>
Group Member
Fri 30 Sep 2022 12:57:51 PM UTC, comment #1: 

This is not CJK-specific but a different internal representation of Unicode text. AFAIK Octave uses UTF-8, meaning that a string of Unicode characters becomes a byte stream. Each of your two Chinese characters is represented in 24 bits of UTF-8 of which 16 bits are content and 8 bits are preset values as described here: https://en.wikipedia.org/wiki/UTF-8#Encoding

You can verify the encoding is correct UTF-8 with these commands:

>> foo = dec2bin ('你' + 0, 8)'(:)'
foo = 111001001011110110100000
>> foo = dec2bin ('好' + 0, 8)'(:)'
foo = 111001011010010110111101
>> foo = dec2bin ('你好' + 0, 8)'(:)'
foo = 111001001011110110100000111001011010010110111101

>> foo = '你好'
>> whos foo
Variables visible from the current scope:

variables in scope: top scope

  Attr   Name        Size                     Bytes  Class
  ====   ====        ====                     =====  =====
         foo         1x6                          6  char


What does Matlab return for those commands? That will tell you what encoding is used internally.

As far as I can see, this is not a bug, unless you get wrong results as a consequence of different Unicode representations.

Arun Giridhar <arungiridhar>
Group Member
Fri 30 Sep 2022 12:28:08 PM UTC, original submission:  

In Octave:

>> str = char("你好")

str = 你好

>> size(str)

ans =
    1    6

We can see that the size of the string of two Chinese characters in Octave is 6.

However, in Matlab:

>> str = char("你好")

str =
    '你好'

>> size(str)

ans =
     1     2
We can see the string of contains two Chinese characters .

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Digest:
   bug dependencies.

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by mmuetzel (Posted a comment)
  • -email is unavailable- added by arungiridhar (Posted a comment)
  • -email is unavailable- added by None (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 3 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2022-09-30 mmuetzel StatusNone Duplicate
        Open/ClosedOpen Closed
        Dependencies- Depends on bugs #49348

    Back to the top

    Powered by Savane 3.13-4448.
    Corresponding source code