bugGNU Octave - Bugs: bug #49348, Treat multi-byte characters as one...


bug #49348: Treat multi-byte characters as one character for char array

Submitted by:  Markus Mützel <mmuetzel>
Submitted on:  Sat 15 Oct 2016 07:30:58 AM UTC  
Category:  Interpreter Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Matlab Compatibility
Status:  None Assigned to:  None
Originator Name:  Open/Closed:  Open
Release:  dev Operating System:  Any

Add a New Comment (Rich Markup)

You are not logged in

Please log in, so followups can be emailed to you.


Sun 23 Oct 2016 07:16:12 PM UTC, comment #4: 

See also bug #43099.
Many of these issues will be solved when MXE builds are made with Qt5 rather than Qt4 together with the patch in bug #49386.

I'll cc John Donoghue as he is the most active mxe-octave dev.

Philip Nienhuis <philipnienhuis>
Project Member
Sat 15 Oct 2016 12:30:49 PM UTC, comment #3: 

You are right, some problems occur in Matlab with Unicode characters that are higher than 65535. Those are clipped to 65535.
So they seem to use UCS-2.
Are there any plans on how to cope with codepage issues in Octave? Should UTF-8 be used everywhere?

Markus Mützel <mmuetzel>
Project Member
Sat 15 Oct 2016 10:17:17 AM UTC, comment #2: 

See also bug #46855 & bug #49222 that undoubtedly motivated this bug report.

I'm sure this has come up in the past several times, though I can't find the bug reports. IIRC it has been mentioned with the symbolic package as well.

There's a relevant wiki page here:
See esp. the paragraphs "The state of Octave" and further.

Philip Nienhuis <philipnienhuis>
Project Member
Sat 15 Oct 2016 09:57:16 AM UTC, comment #1: 

Octave uses one byte for char arrays.  Matlab uses wchar_t or 2 bytes for each character.  I believe you can generate the same problems with Matlab if you have a UTF-32 character.

I've marked this as postponed.  This would be a very deep change to the internals of Octave.

Rik <rik5>
Project Administrator
Sat 15 Oct 2016 07:30:58 AM UTC, original submission:  

In Octave, one char seems to be one byte long always. This leads to the following issues when reversing or indexing into a character array with multi-byte characters:


In Octave:

>> t='abc\303\244\303\266\303\274'
t = abcäöü
>> t(end:-1:1)
ans = �öä�cba
>> size(t)
ans =
   1   9
>> t(4:6)
ans = ä�

In Matlab:

>> t='abcäöü'
t =
>> t(end:-1:1)
ans =
>> size(t)
ans =
     1     6
>> t(4:6)
ans =

I also noticed that char() doesn't support values >255 and messes up Unicode characters >127:
In Octave:

>> char (269)
warning: range error for conversion to character value
ans =
>> char(228)
ans = �

In Matlab:

>> char(269)
ans =
>> char(228)
ans =

Is this a design choice in Octave? Would it be possible that Octave's char class treated Unicode characters as such no matter how many bytes they use in any encoding (UTF-8 seems to be a good choice)?

Markus Mützel <mmuetzel>
Project Member


(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:

No files currently attached


Depends on the following items: None found

Items that depend on this one: None found


Carbon-Copy List
  • -email is unavailable- added by philipnienhuis
  • -email is unavailable- added by rik5 (Posted a comment)
  • -email is unavailable- added by mmuetzel (Submitted the item)

    Do you think this task is very important?
    If so, you can add your encouragement to it.
    This task has 0 encouragements so far.

    Only project members can vote.




    Follow 3 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2016-10-23 philipnienhuis Carbon-Copy- => Added lostbard
    2016-10-15 philipnienhuis StatusPostponed => None
    2016-10-15 rik5 StatusNone => Postponed

    Back to the top

    Powered by Savane 3.5