bugGNU Octave - Bugs: bug #58637, Fullwidth CJK characters are...

 
 

bug #58637: Fullwidth CJK characters are displayed twice in command window (on Windows)

Submitted by:  yan <pop>
Submitted on:  Sun 21 Jun 2020 03:27:37 PM UTC  
 
Category:  GUI Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Unexpected Error or Warning
Status:  Confirmed Assigned to:  None
Originator Name:  yan Open/Closed:  Open
Release:  dev Operating System:  Microsoft Windows

Add a New Comment (Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

( Jump to the original submission)

Thu 29 Oct 2020 07:53:51 AM UTC, comment #8: 

Moving the release to "dev" because fixing this probably relies on having a cross-platform terminal widget. That is very likely a huge project.

Markus Mützel <mmuetzel>
Project Member
Mon 21 Sep 2020 08:35:14 PM UTC, comment #7: 

I forgot to mention, the second result in comment #6 is for the cli version.

Torsten Lilge <ttl>
Project Member
Mon 21 Sep 2020 07:11:10 PM UTC, comment #6: 

Here is the result

>> __mfile_encoding__
ans = utf-8
>> __locale_charset__
ans = CP1252
>> __mfile_encoding__ utf-8
>> clear faa
>> faa
e = è‘—
ans = è‘—

Thus, it looks like the problem is the limited support for utf-8 in windows 7.

octave:6> __mfile_encoding__
ans = system
octave:7> __locale_charset__
ans = CP1252
octave:8> __mfile_encoding__ utf-8
octave:9> clear faa
octave:10> faa
e = Þæù
ans = Þæù

Torsten Lilge <ttl>
Project Member
Mon 21 Sep 2020 06:28:18 PM UTC, comment #5: 

@Torsten: That looks like the file was parsed as ISO 8859-1 (or a similar encoding like CP-1251 or others). What do you get for `__mfile_encoding__`? If it is "system", what does `__locale_charset__` return?
That file is encoded in UTF-8. You might need to change the `__mfile_encoding__` and reparse the file:

__mfile_encoding__
__locale_charset__
__mfile_encoding__ utf-8
clear faa
faa

Might also be a difference between Windows 10 and Windows 7. I believe Unicode support at the command prompt was still incomplete in Windows 7.

Does Windows 7 have a "Legacy Console" setting? In Windows 10 that setting can be changed in the properties of the command prompt (cmd.exe). The "Options" tab has a "Use legacy console" checkbox which is unset for me. (I translated those settings from German. So their actual name might be different.)
IIRC that setting also effects how the Octave command window behaves in Windows.

Markus Mützel <mmuetzel>
Project Member
Mon 21 Sep 2020 06:10:12 PM UTC, comment #4: 

Torsten: might depend on the default "legacy" (non-Unicode) code page the two of you have defined for your respective systems?

Andrew Janke <apjanke>
Mon 21 Sep 2020 06:06:08 PM UTC, comment #3: 

On my VM with Widnows 7 I get

>> faa
e = è‘—
ans = è‘—

in the GUI. What might be the difference?

As a side effect, I noticed that changing the default file encoding in the preferences only takes effect after restarting octave on Windows.

Torsten Lilge <ttl>
Project Member
Mon 29 Jun 2020 02:43:01 PM UTC, comment #2: 

Does sound like an issue with the Windows-specific command window library. Works fine (no doubled CJK characters) in the 4.4.1 GUI on Mac for me.

Andrew Janke <apjanke>
Sun 21 Jun 2020 04:56:21 PM UTC, comment #1: 

Are you referring to the fact that each Chinese character is displayed twice in the Command Window of the GUI on Windows?

In that case, I can confirm the issue using the GUI with the attached file (from bug #50404).
Output in the command window:

>> cd D:\Repositories\Octave\bug58637
>> dir
.       ..      faa.m   äöü  著著
>> faa
e = 著著
ans = 著著

I can't reproduce with the CLI. After setting the .m file encoding and terminal codepage to UTF-8, output seems to be ok here:

octave:1> cd D:\Repositories\Octave\bug58637
octave:2> system("chcp 65001")
Aktive Codepage: 65001.
ans = 0
octave:3> __mfile_encoding__ utf-8
octave:4> clear functions
octave:5> dir
.       ..      faa.m   äöü  著
octave:6> faa
e = 著
ans = 著
octave:7>

This is probably an (upstream?) issue with the command window implementation in the GUI on Windows.

(file #49339)

Markus Mützel <mmuetzel>
Project Member
Sun 21 Jun 2020 03:27:37 PM UTC, original submission:  

CLI browses the contents of the current directory, Chinese characters overlap

yan <pop>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #49339:  faa.m added by mmuetzel (36B - application/octet-stream)
file #49337:  bug.png added by pop (60KiB - image/png)

 

Depends on the following items: None found

Digest:
   bug dependencies.

 

Carbon-Copy List
  • -email is unavailable- added by ttl (Posted a comment)
  • -email is unavailable- added by apjanke (Posted a comment)
  • -email is unavailable- added by mmuetzel (Updated the item)
  • -email is unavailable- added by pop (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can add your encouragement to it.
    This task has 0 encouragements so far.

    Only project members can vote.

     

     

     

    Follow 8 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2020-10-29 mmuetzel Release5.2.0 => dev
    2020-09-01 siko1056 Dependencies- => bugs #58834 is dependent
    2020-06-21 mmuetzel Attached File- => Added faa.m, #49339
        CategoryNone => GUI
        Item GroupNone => Unexpected Error or Warning
        StatusNone => Confirmed
        SummaryCLI browses the contents of the current directory, Chinese characters overlap => Fullwidth CJK characters are displayed twice in command window (on Windows)
    2020-06-21 pop Attached File- => Added bug.png, #49337

    Back to the top


    Powered by Savane 3.5