Wed 19 Jul 2017 08:43:54 AM UTC, comment #40:
@Markus. I understand. Thank you, Markus for answer!
It's not need you, but there is exist - atached screenshot from Octave-utf8-editor in WinXP
(file #41241)
|
Wed 19 Jul 2017 08:30:24 AM UTC, comment #39:
Octave uses the Windows console for its Command Window (on MS Windows). That is the same console that is used by "cmd.exe".
Unfortunately, its support for the "UTF-8 codepage" 65001 is broken in Windows versions prior to Windows 10.
The only way to fix this in Octave would be to completely re-design how Octave handles input/output at the command line (cross-platform). This major project is on the wish-list. But nobody is working on it at the moment as far as I know.
|
Wed 19 Jul 2017 06:31:05 AM UTC, comment #38:
I forgot say: from utf-8 Octave's editor (menu "Edit" -> "Preferences")
|
Wed 19 Jul 2017 06:20:47 AM UTC, comment #37:
I'm sorry I am connected to this branch. (@Markus: thank you very much you found this branch of bug to me)
I confused because text read good at Win10 and not read at Win7,Win8.
Do I correct understood this bug still is not solved on win7-8?
I attached my pictures:
(file #41239, file #41240)
|
Thu 23 Mar 2017 11:34:15 AM UTC, comment #36:
Renaming to the title proposed by Hartmut in bug #49386 to better reflect the remaining issue.
|
Sun 13 Nov 2016 11:03:58 AM UTC, comment #35:
This bug report could be renamed to something like "Non-ASCII characters not displayed correctly in command window for Windows 7 and earlier" to reflect the remaining issue.
I guess we can leave it open until someone with Windows 7 seems urged to work on it.
|
Tue 25 Oct 2016 08:54:08 PM UTC, comment #34:
Hmmm I lost track of where to post comments as regards these character encoding stuff. So many bug reports about encoding & character display ...
With Hartmut's patch (file #38798) in bug #49118, qt4 and Markus' patch (file #38784) here in bug #43099:
- on Win 10 the GUI terminal properly displays UTF8 input from Andreas' .xlsx file in bug #49222 (file #38622)
- on Win7 however the terminal is still broken. As I wrote I think it is a problem with Win7's cmd.exe. When encoding UTF-8 input through Markus utf82unicode.m (see bug #49222) the contents are displayed properly.
My conclusion is that the display of non-ascii in Windows 7 is broken beyond repair. JWE wrote in another bug report that he hopes to have the terminal replaced by a pane/widget independent of bash or cmd.exe. So it may be better to wait for that to happen.
If you want I can upload the qt4 and qt5 builds (both have the same patches) to my dropbox. Send me a private mail if you want the URLs.
|
Fri 21 Oct 2016 07:24:31 PM UTC, comment #33:
With the patch,
- On Windows 10 I get perfect results, also in the GUI terminal (even w/o the system ("chcp 65001") command)
- on Windows 7, still no luck as far as the GUI terminal goes.
So there's a difference between Win7 and Win10 :-(
Win7 will be supported until Jan. 14, 2020, so something needs to be done for the Windows builds I suppose.
But at least for the symbolic package it can be tried to get "beauty" like for Linux (see comment #14 :-) )
|
Fri 21 Oct 2016 03:46:56 PM UTC, comment #32:
On Win 10, with 4.3.0+ before your path, some results are better.
mice_and_rice look OK, see attached pic.
ellipsize, lquot and rquot look wrong.
I'm waiting for the cross-build to complete & install it.
(file #38786)
|
Fri 21 Oct 2016 03:17:05 PM UTC, comment #31:
Windows 7.
I can try on Win10 as well, both my desktop & laptop multiboot Win7/Win10/Mageia-5
|
Fri 21 Oct 2016 10:37:23 AM UTC, comment #30:
Philip,
Thank you for testing.
It looks like the codepage did not really change for you and the console is still using cp 1252. At least, I get the same output as you with that codepage.
Could you please try with the attached patch? That should set the codepage to UTF-8 (65001) when the console is created.
I am testing on Windwos 10 64bit. Which version of Windows are you using?
(file #38784)
|
Thu 20 Oct 2016 06:22:48 PM UTC, comment #29:
Unfortunately I can't confirm that qt5 solves all of it, that is, the terminal is still broken.
I also cross-built Octave-4.3.0+ (--enable-windows-64) with qt5.
Running 'system ("chcp 65001")' and then running testing_utf8.m (using F9 from the editor) gives the attached screenshot.
In the workspace, editor, and history pane the strings come out fine.
The editor merely needs "UTF-8", that also works fine with Qt4.
The terminal and Variable Editor (patch #8656) show garbage.
(Now the Variable Editor is highly experimental, but on Linux it works fine as well as regards encoding.)
(file #38777)
|
Sat 15 Oct 2016 06:50:57 AM UTC, comment #28:
After some tests (see below), I'd like to propose to change the default codepage for the editor and the command window to UTF-8 on Windows and distribute a version that is compiled against QT5.
I cross-compiled Octave with QT5 and most of the issues seem to be resolved after I change the following manually:
- Change encoding of the editor from "SYSTEM" to "UTF-8" in the properties.
- Execute 'system ("chcp 65001")' in the command window.
After this, when I run Colin's "testing_UTF8.m", I get the following correct output:
The characters also display fine in the "Workspace" view. They look weird in 4.2.0-rc2.
Selecting all and pressing F9 yields the following:
This looks like the decoded bytestream in UTF-8.
I still cannot enter umlauts like "äöü" or "ß" in the command window (bug #47571). The keystrokes to these letters are silently ignored. Just a wild guess because I don't know how that actually works: Could this be an issue with readline on Windows which drops non-ASCII characters?
|
Sun 17 Jul 2016 01:04:46 AM UTC, comment #27:
I noticed that non ascii characters saved as utf8 are not compatible between on 4.0.x and 4.1.0+ on the octave editor.
I can transfer them using other editors (e.g notepad++) via clipboard.
Does some other person see this?
|
Wed 13 Jul 2016 07:58:03 AM UTC, comment #26:
I propose that
1. bug #44538 and bug #47571 integrated to one
(omit #47571) and title of #44538 is changed to
"command window problem for windows for non-ASCII characters"
2. bug #43099 (this bug) and bug #48306 integrated to one
and remove command windows problem and re-titled.
(I do not find good title at present.
Colin, do you have a good title?)
|
Wed 13 Jul 2016 03:10:27 AM UTC, comment #25:
For the bug #44538 and bug #47571 are duplicated to this bug on command window issue. However, the command windows issue is intrinsically different from others because it is related to windows console but others are not. The issue for command windows should be treated exasperatedly.
The bug #48306 is related to documentation windows.
The bugs are categorized in two.
1. Issues purely comes from qt-qui.
(editor, workspace pane, and documentation windows)
2. Issue on command windows which dependes both on qt-gui and command console of windows.
|
Tue 12 Jul 2016 07:12:46 PM UTC, comment #24:
You didn't mention the documentation browser. Can you test that as well? Some help strings should have some UTF-8 quote characters or bullets.
Can someone look at related bugs and see if there is anything additional there or if they can be closed as duplicates of this report?
Check out bug #44538, bug #47571, and bug #48306.
|
Sun 03 Jul 2016 06:34:01 AM UTC, comment #23:
Anyway, I agree the critical thing is the command windows. +1 for having this on the 4.2 list.
I just tried again on 4.0.3. What I see is different from the title (but also slightly different from Tatsuro:
Editor: utf-8 looks fine (Chinese and some math chars tested)
Command windows: broken (usual high ascii junk)
Workspace pane: broken (but with different chars). In my original report, this worked.
I think this on Windows 10. Its a different machine than I originally reported this bug on.
|
Sat 02 Jul 2016 09:15:56 AM UTC, comment #22:
Is the title correct?
utf-8 files: editor, command windows issues, workspace pane shows correctly
For editor, Japanese characters encoded in utf8 are represented correctly if the font is set properly on "preferences".
I think that the issue is limited to the command window.
|
Sun 22 Mar 2015 01:10:45 AM UTC, comment #21:
I had seen that - lots of "in windows you will need to do xxxx and yyyy and ..... "
|
Sun 22 Mar 2015 12:47:41 AM UTC, comment #20:
> Windows seems to like utf-16, so maybe all we need to do is
> wrapper any functions to convert between utf-8 and 16.
Is this helpful? http://utf8everywhere.org
|
Sun 22 Mar 2015 12:41:16 AM UTC, comment #19:
Kind of hard to tell if its still an issue, but many reports where setting the code page for UTF-8 breaks OS level functions such as ReadFile etc.
Windows seems to like utf-16, so maybe all we need to do is wrapper any functions to convert between utf-8 and 16.
|
Sat 21 Mar 2015 11:35:54 PM UTC, comment #18:
@Colin:
No I wasn't offended at all of course, in fact I appreciated the joke. Seriously, there's quite some truth in it....
@JohnD:
Is it really true that "unicode support on Windows is broken at OS level"? I'd rather suspect it's the mingw tools that are at fault. On e.g. stackoverflow there are quite a few reports on unicode issues with mingw.
AFAIK MSVC++ doesn't have that many unicode problems.
|
Sat 21 Mar 2015 06:18:31 PM UTC, comment #17:
@Philip: I meant no offense and hope none was taken. It is definitely important to have Octave working on Windows.
IMHO, the title of this bug is correct:
* On Windows, workspace pane shows utf-8 strings correctly.
* On Windows, editor and command terminal show gibberish
for utf-8 encoded files/strings.
(On GNU/Linux, both are fine.)
@lostbard: understood. I will leave the windows-ascii-default in place in the Symbolic pkg.
|
Sat 21 Mar 2015 11:21:40 AM UTC, comment #16:
There is now a collection of Windows utf-8 related bugs reported.
Some for display/font issues, some for file/os related functions.
And indications that windows unicode support is broken at the OS level.
So ... its probaly going to take a while to fix the issues.
|
Sat 21 Mar 2015 10:05:24 AM UTC, comment #15:
"on GNU/Linux we have beauty. Which is of course true on many levels"
Sure, point taken :-)
[OT] (BTW I'm no windows fan or so. I just happen to work in a "culture" where Windows (and much other proprietary SW) is used for about everything; if they could make coffee with it they'd do that as well. So my primary interest here, apart from fun/hobby, is simply to get/keep Octave running in that environment/culture. No more, no less.
[Back to topic]
So, had this bug better be renamed to "Unicode doesn't work on Windows"? Or just leave it as is, enter a new bug report for unicode-on-windows (if it isn't already there) and add that as a dependency? (that seems to be possible)
What I will do is set "Release" to "dev"
|
Thu 19 Mar 2015 10:06:03 PM UTC, comment #14:
> Is it all just a matter of a sympref setting? if so it might be easy to tackle this.
The point of this bug is that unicode doesn't work on Windows.
If I understand correctly, what you are suggesting is already done in symbolic: I detect Windows and default to ugly ascii art there to workaround this bug. Whereas on GNU/Linux we have beauty. Which is of course true on many levels ;-)
|
Thu 19 Mar 2015 09:54:29 PM UTC, comment #13:
Sorry for a lte reply (busy week).
With "sympref display unicode" I get the garbled stuff back.
AFAIU unicode isn't supported in the Octave terminal?
But on Windows with default settings it all works fine.
Is it all just a matter of a sympref setting? if so it might be easy to tackle this.
|
Wed 18 Mar 2015 10:37:33 PM UTC, comment #12:
My collaborator Andrés says:
> Unfortunately, Octave 4.0.0-rc1 fails to handle unicode
> strings correctly (see the attached screenshot). It has been
> tested using Windows 7 Professional (Service Pack 1).
I will attach a crop of that screenshot. So no, we're not there yet.
(file #33395)
|
Mon 16 Mar 2015 01:46:54 PM UTC, comment #11:
I noticed the windows binary from ftp://alpha.gnu.org/gnu/octave/ is an ...-installer.exe. I just tried and I do not have permissions on the terminal server at work to install this. @Philip: are you able to test this? Sorry for the hassle.
|
Sun 15 Mar 2015 10:24:28 PM UTC, comment #10:
Philip, thank you for testing but unfortunately that doesn't really check, because I changed the default to ascii on windows (b/c of this issue).
I'll try to test early this week. But if easy for you to do so, try redoing your example with "sympref display unicode" (the default on unix).
|
Sun 15 Mar 2015 10:10:26 PM UTC, comment #9:
With 4.0.0-rc1 & symbolic-2.1.0 I see no more garbled text on the terminal (see attached pic).
Depending on used fonts, several characters used to draw e.g., large parentheses, might not line up very perfectly, but that's a minor issue.
All in all the symbolic package is very usable on Windows now. Thanks!
(file #33356)
|
Sat 14 Mar 2015 07:54:19 AM UTC, comment #8:
Colin, could you please try the 4.0.0-rc1 build? The changeset http://hg.savannah.gnu.org/hgweb/octave/rev/3db04b75c7c0?revcount=40 might have fixed some of the issues.
|
Mon 09 Mar 2015 12:07:42 AM UTC, comment #7:
Sorry not sure, but almost certainly they are related.
|
Sun 08 Mar 2015 02:44:42 PM UTC, comment #6:
Is there any difference between this bug and bug #42036?
|
Wed 18 Feb 2015 09:10:29 PM UTC, comment #5:
Comment #4 implies that the mxe-build of octave (not the gui) for windows is ansi not utf-8. Is there an easy way to test an unicode build?
|
Tue 10 Feb 2015 09:07:49 PM UTC, comment #4:
I noticed that the command history has the encoding issue, too, but also noticed that this issue disappears if I manually change the encoding of the .octave_hist file in my personal directory to UTF-8 (via Notepad++, ...). So a really minor fix to one the bugs related to the encoding issues would be to make sure the encoding of the generated .octave_hist file is actually UTF-8. (Then the filter works, too). Of course this does not solve the original bug report but since the report is rather general, I posted my observations here.
See screenshots below for a simple demonstration.
(file #33047, file #33048)
|
Thu 28 Aug 2014 11:38:58 PM UTC, comment #3:
Yes, that is why I marked it Windows :)
|
Thu 28 Aug 2014 08:31:00 PM UTC, comment #2:
Works for me on linux in the gui- and cli-version of the gui-release branch.
The font in the editor can be changed in the settings dialog (file menu / preferences) and there in the editor-styles tab. But I guess this won't change anything.
|
Thu 28 Aug 2014 10:04:18 AM UTC, comment #1:
bug #42036 probably related.
|
Thu 28 Aug 2014 10:00:49 AM UTC, original submission:
See attached screen shot.
Could be a font issue, although I tried changing the command windows from "Courier New" to "DejaVu Sans Mono" (the font I use on GNU/Linux) and it looks essentially the smae (see 2nd screenshot). I didn't see a way to change the font in the editor.
Its also broken in the "cmd.exe" terminal.
.m file attached.
|