bugGNU Octave - Bugs: bug #49118, GUI editor cannot run file in...

 
 

bug #49118: GUI editor cannot run file in folder name with umlaut

Submitted by:  Hartmut <hardy>
Submitted on:  Sat 17 Sep 2016 08:57:00 PM UTC  
 
Category: GUISeverity: 3 - Normal
Priority: 5 - NormalItem Group: Incorrect Result
Status: Patch SubmittedAssigned to: None
Originator Name: Open/Closed: Open
Release: devOperating System: Microsoft Windows

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

(Jump to the original submission Jump to the original submission)

Wed 18 Oct 2017 11:18:33 AM UTC, comment #53:

@jwe: the patch now fulfills the requirements you had in comment #44. Do you think it can be pushed now? As a french user, I can say that being able to operate on files or folders with non ascii characters is very important.

Pantxo Diribarne <pantxo>
Project Member
Mon 16 Oct 2017 06:15:04 PM UTC, comment #52:

The attached version of the patch also adds support for opening a pipe with non-ASCII characters on Windows (to fix bug #52211).

(file #42167)

Markus Mützel <mmuetzel>
Project Member
Sat 26 Aug 2017 07:48:42 PM UTC, comment #51:

Thank you for testing, Avinoam.

The attached patch also adds support for opening files with non-ASCII characters in functions that use std::ifstream. That includes dlmread (bug #51836).
I haven't found a portable solution for writing to a new file with std::ofstream (other than using the C++17 std::filesystem::path which is very new or Boost).

(file #41670)

Markus Mützel <mmuetzel>
Project Member
Tue 22 Aug 2017 01:29:14 PM UTC, comment #50:

I have checked this patch on Win-10 machine with folder that contains Hebrew letters.
In 4.2.1, when trying to change directory to this directory, the result is "parse error", on 4.3.0+ with the patch it works fine.

Thanks!

Avinoam Kalma <avinoam>
Project Member
Thu 03 Aug 2017 02:04:50 PM UTC, comment #49:

The last patch no longer applies again. The attached new patch also fixes a failing test with the last one.

(file #41401)

Markus Mützel <mmuetzel>
Project Member
Wed 19 Jul 2017 09:11:03 PM UTC, comment #48:

Upps. :-)
Next attempt.

(file #41251)

Markus Mützel <mmuetzel>
Project Member
Wed 19 Jul 2017 09:04:12 PM UTC, comment #47:

I don't see a new patch :-)

Maybe the patch could be in mxe-octave?

Philip Nienhuis <philipnienhuis>
Project Member
Wed 19 Jul 2017 06:09:14 PM UTC, comment #46:

The patch in file #39829 no longer applies to a current tip. Attached is a refreshed patch that also adds support for NTFS variants that do not store short (8.3) filenames (like my new laptop with pre-installed Win10).

Is there any interest in taking this patch that only helps the user experience on Windows?
There are no new #if defined (OCTAVE_USE_WINDOWS_API) conditionals (other than in sysdep.cc and lo-sysdep.cc).

Markus Mützel <mmuetzel>
Project Member
Sat 25 Feb 2017 01:49:36 PM UTC, comment #45:

As far as I can tell, gnulib won't use the wide character API on Windows making it impossible (or very un-reliable) to access files and folder with non-ASCII characters. One would have to convert the internal UTF-8 to the locale specific code page which would loose characters that cannot be represented in that codepage.
Using boost::Filesystem [1] could help since it seems to use the wide character API internally. But that would add a huge new dependency to Octave and need a (major?) re-write of the functions handling the file system access. Would that be acceptable and preferred?

Meanwhile, I found that uniconv accepts "wchar_t" as a code page string. This helps a lot when it comes to handling the different sizes of wchar_t on different platforms and its corresponding encoding (UTF-16 vs UTF-32). And hence eliminates the use of the Windows specific conversion functions.
I also moved all system dependent functions to the sysdep files.
Please, see the attached patch.

[1]: http://www.boost.org/doc/libs/1_42_0/libs/filesystem/doc/index.htm

(file #39829)

Markus Mützel <mmuetzel>
Project Member
Tue 31 Jan 2017 07:29:41 PM UTC, comment #44:

I'd prefer to find a way to remove #if defined (OCTAVE_USE_WINDOWS_API) conditionals, not add more.

We already have system-dependent functions in gnulib and sysdep files in liboctave and libinterp. I think we should just add system-dependent functions there if they are needed.

John W. Eaton <jwe>
Project Administrator
Mon 30 Jan 2017 10:40:46 PM UTC, comment #43:

With the attached patch Octave also no longer crashes in Windows when calling dir in a directory with non-ASCII characters.

It also no longer needs a newer compiler since I went back to using the conversion function supplied by the WinAPI. I wrapped those calls in functions in a new header oct-winapi.h which must only be included for the Windows API. Is this conceptionally OK?

(file #39620)

Markus Mützel <mmuetzel>
Project Member
Tue 06 Dec 2016 08:15:45 PM UTC, comment #42:

Attached is a patch in which I got rid of the pragma conditionals in the header files. I also added wrappers so that these conditionals are no longer needed in load-save.cc and the parser.

The conversion between byte strings (UTF-8, what Octave uses) and wstrings (UTF-16, what the Windows filesystem uses) is done in u8_to_wstring and wstring_to_u8 which converts between std::string and std::wstring with the respective encodings.

However, it now only compiles with gcc 5 or newer because it uses the C++11 function std::wstring_convert. I tried with gcc 5.4.0 in mxe-octave, and it works as expected for me on Windows 10.

I am still not sure which conditionals should be used. I stuck with OCTAVE_USE_WINDOWS_API. But I have also seen OCTAVE_HAVE_WINDOWS_FILESYSTEM and ! OCTAVE_HAVE_POSIX_FILESYSTEM in other places. I do not know the build system well enough to judge which should be used here...

(file #39166)

Markus Mützel <mmuetzel>
Project Member
Tue 29 Nov 2016 09:46:53 PM UTC, comment #41:

No. The version of gcc in mxe-octave was tied up with the 4.2.0 release and wanting to be a bit conservative. I'm sure the next major version of Octave (4.4.0) will use a newer version of gcc.

Rik <rik5>
Project Administrator
Tue 29 Nov 2016 08:42:09 PM UTC, comment #40:

"std::codecvt_utf8_utf16" and "std::wstring_convert" require at least version 5 of gcc. At the moment, we use gcc 4.9.4 in mxe-octave. Is there a reason why we don't use a newer version?

Markus Mützel <mmuetzel>
Project Member
Wed 23 Nov 2016 05:12:09 PM UTC, comment #39:

Thank you very much for the feedback, Mike.

It was quite straight forward removing the C wrappers and introducing the abstaction layer for fopen (see attached patch).
However, I am at a loss where the functionality for getting a UTF-16 wchar_t from an Octave string should go. It would be nice to have something like "wc_str" that would return a pointer to a wchar_t string corresponding to the multi-byte string in std::string (like "c_str" does directly for char). Is that what you meant? Where should something like that go? In the octave_char_matrix class?
For the 8.3 paths: Is there already a class in Octave that is used for representing file paths?

The conditionals in the file-ops header (still present) won't be needed anymore when these two abstractions are implemented, I think.

I found that there is std::codecvt_utf8_utf16 in C++11. Thus, there is probably an easy way to not rely on the WinAPI for the conversion between UTF-8 (char, string) and UTF-16 (wchar_t, wstring). Is it OK to use features that are not present in earlier versions of Standard C++?

(file #39058)

Markus Mützel <mmuetzel>
Project Member
Thu 17 Nov 2016 04:38:41 AM UTC, comment #38:

Retagging to 4.2.0 release.

I had a look at the patch, and even though I can't test or build with this, I suppose if this is the only way to get wide character file name support in Windows this is a reasonable approach. I think it might be cleaner if we could more clearly define the layer between the type that Octave uses for strings (now a std::string, but in the future it may become a std::wstring) from what the OS (or other libraries) expect to be given to represent the name of a file/directory/resource.

I don't think you need to add new wrapper functions for these Windows library calls in the liboctave/wrappers directory. The purpose of that directory is to encapsulate the workarounds between glibc and gnulib into a small C API that Octave's C++ libraries can use unconditionally. If you're going to make the code elsewhere dependent on whether it's Windows or not, and whether the function takes a char or a wchar_t, then you might as well just call the Windows function directly. For exmaple, instead of defining a new function octave_wmkdir_wrapper, just use an #if in place and call _wmkdir directly.

It would be nice if the workarounds that have to be done for HDF5 could be abstracted away a little more so the load-save.cc module doesn't have to know details about which OS it is running on, just about how to get a safe file name from an Octave string type.

Same thing with the parser, maybe we can add an fopen wrapper to liboctave/system/file-ops.cc to encapsulate calling either std::fopen or _wfopen.

Definitely avoid putting conditionals into public header files, the OCTAVE_USE_WINDOWS_API can't be put into the file-ops.h header file.

Mike Miller <mtmiller>
Project Administrator
Sun 13 Nov 2016 01:48:35 PM UTC, comment #37:

Attached is a patch that should be working a lto better. The issues mentioned in comment #35 no longer occur.
It uses the wide character API in Windows where possible which enables the access to directories with non-ASCII characters. I had to resort to short 8.3 filenames for the H5F part because there is no wide character interface to it.

I am not really sure where the boundaries of Octave are. Please, let me know if something needs to be changed. Maybe move the conversion to wide characters (and back) into the wrappers?

This might also solve part of bug #42036 (directories with non-ASCII characters). CC'ing Tatsuro because he seems to have worked on that bug, too.

(file #38945)

Markus Mützel <mmuetzel>
Project Member
Mon 07 Nov 2016 07:14:27 PM UTC, comment #36:

This bug won't stop the 4.2 release, but it is going to be a hard one to fix. Usually one wants to convert encoding, if necessary, at the boundaries of the application and then use whatever encoding was chosen internally in a consistent manner. But Octave's insides and outsides are all jumbled so that there are places in load/save, in the GUI, in the LOADPATH, etc. where the encoding matters.

Rik <rik5>
Project Administrator
Mon 07 Nov 2016 06:26:31 PM UTC, comment #35:

There is still a lot more to do. Even if it is now possible to change to a directory with Umlauts, there are still problems executing files in there. Executing the un-modified file works. However, when I modify and save it in the editor, I get:

Where yyy is the path to the mxe-repo in Linux. The folder I was testing with was "D:/SVN/Octave/Ümläüt".

The change in the file is ignored (the old file is executed).
After I do a "path(path)" which normally helps when Octave does not pick up changed files, I get a dialog with:
"The file D:/SVN/Octave/Ümläüt/testing_utf8_2.m is shadowed by a file with the same name in the load path. To run or debug the function you are editing, change to the directory D:/SVN/Octave/Ümläüt."
When I click on "Change Directory", the same dialog pops up again.
Oddly, this happened after I ran the test file from the command window.

Additionally, the path is not displayed correctly in the "Current Directory:" dropdown list.

I guess the results from "dir" have to be converted from UTF-16 to UTF-8 as well?

Seeing all this, I'd like to propose that we don't block 4.2 on this issue (even if it is marked as a regression). Working with non-ASCII characters on Windows never really worked in Octave. Some aspects improved, some aspects got worse since the last release. But I think we are on the right track with accepting patches which move us towards "UTF-8 everywhere" even on Win32.

Having said that, I would like to continue working on that bug if someone could give me some help.

Markus Mützel <mmuetzel>
Project Member
Mon 07 Nov 2016 05:31:16 PM UTC, comment #34:

Attached is a work in progress patch which enables changing to a directory with non-ASCII chars in Windows for me (feature A from comment #32).

I think this was working for Umlaut characters before file #38798 (the "cstrings-patch") was applied because our system codepage coincidentially was latin-1 and we only tried with Latin-1 characters.

The patch converts the UTF-8 encoded path to UTF-16 and calls the wchar_t variant of chdir (_wchdir) for Windows.

This is still "work in progress" because I definitely have the #if-conditions wrong. Can someone please help me figure out which constants should be checked (UNICODE, OCTAVE_USE_WINDOWS_API, ...)?
And more importantly: Is this the right way to do it? Most probably "mkdir", "rmdir" and other functions that access the file system would need special treatment for Win32 as well...

(file #38898)

Markus Mützel <mmuetzel>
Project Member
Wed 02 Nov 2016 06:50:24 PM UTC, comment #33:

small addition to comment #32:
The feature BCD only works in Win7 with Octave 4.2.0-rc3 if you change the editor SETTING for text encoding to UTF-8. The default value here is "System" which lets the features BCD still fail.

side question: Would it be a good idea to change the default value of the GUI editor's text encoding to utf8? (Already existing m-files would still be opened in their existing text encoding. This would just be a change for newly created m-files.) This change might even be useful for linux octave, (maybe even necessary, I have never tested in linux with a different system setting than utf8.)

Hartmut <hardy>
Tue 01 Nov 2016 09:17:33 PM UTC, comment #32:

I have done some (even more extensive) testing. (Maybe I should have done this before, but I wasn't expecting those side effects of my patch).

I have tested following FEATURES:

  • A) run a m-file script in a folder with an umlaut in its name
  • B) get the correct return value from inputdlg when typing an umlaut into its window
  • C) displaying the correct "question" text string in the inputdlg window, when this string contains an umlaut
  • D) display a string array correctly in the GUI variable explorer, if this string array contains an umlaut. Display at least some other UTF8-characters properly as well.

The features B+C+D only seem to work all or none. Feature A works or fails independently.

Here are my TEST RESULTS (all with qt4, Windows = Win7 and WinXP):

  • Octave 4.0.3 release:
    • Linux: A works, BCD works
    • Windows: A works, BCD fails
  • Octave 4.2.0-rc2:
    • Linux: A fails, BCD fails
    • Windows: A works, BCD fails
  • Octave 4.2.0-rc3:
    • Linux: A works, BCD works
    • Windows: A fails, BCD works
  • A have strong evidence that only applying the "cstrings-patch" (xx) to rc2
    • with Linux: swaps ABCD from "fails" to "works"
    • with Windows: swaps A from "works" to "fails", but also swaps BCD from "fails" to "works"
  • I have tested the "toLocal8BitPatch" (file #38861) and it does NOT help to make A work under Windows.

My CONCLUSION: So with the current rc3 the only failure is feature A under Windows. But none of the already discussed (text encoding related) patches helps. And undoing the "cstrings-patch" (comment #19) only for Windows would destroy the features BCD under Windows. (What is more important to have under Windows A or BCD?)
But this current (rc3) failure of feature A under Windows is still a regression, it used to work under Octave 4.0.3.

Does this (failing A = running scripts in umlaut folders) also fails for mxe-octave 4.2.0-rc3 built with the newly introduced qt5 option? Do those qt5 mxe-builts support features BCD?

Hartmut <hardy>
Tue 01 Nov 2016 12:34:18 PM UTC, comment #31:

Ooops. We have improved this behavior under Linux (qt4) BUT instead we have now created the very same problems under Windows (where this aspect was fine with rc2 before).

I have tested Octave-4.2.0-rc3:

  • Under Linux (self built from official release tar-ball with qt4) both mentioned behaviors of this bug report are FINE now: I can run m-files in folders with umlaut. And I can type umlauts in a dialog window of inputdlg, they are afterwards returned and displayed correctly.
  • But on Win 7 things got worse with our changes between rc2 and rc3: The return value of inputdlg is NOT correct now if I type umlauts in this dialog windows. And I canNOT run m-files in folders with umlaut names any more.

So our changes between Octave 4.2.0-rc2 and rc3 seem to just have pushed this problem over from linux octave to windows octave :(

What do we do about this? Should we introduce even more compiler conditionals like "if Win32 then use file #38294"? But this patch was also relevant for bug #49386, and for those issues over there (mostly umlauts in Octave created windows) this patch still DOES help. Or shall we revert those changes because in linux there is at least an alternative (qt5) that is Windows does not yet exists?

Hartmut <hardy>
Thu 27 Oct 2016 05:53:36 PM UTC, comment #30:

I've just compiled the current stable branch. And the original issue is now fixed for me on linux (I have only ever observed this specific issue on linux. Nevertheless this patch also improves the behavior of mxe-octave with non-english charcters on Windows as well.)

I suggest to close this bug report now as fixed. The discussion about improvements to the qt5 version of mxe-octave could be continued elsewhere (with an appropriate title). Other issues with non-english characters are dealt with in several seperate bug reports.

Hartmut <hardy>
Thu 27 Oct 2016 05:16:30 PM UTC, comment #29:

I pushed your change to stable:

http://hg.savannah.gnu.org/hgweb/octave/rev/f88de8f27259

Can this bug report be closed as fixed?

John W. Eaton <jwe>
Project Administrator
Wed 26 Oct 2016 07:48:44 PM UTC, comment #28:

I think that comment #1 and Rik's comment #8 are saying that linux builts with qt5 do NOT have this problem.

My tiny patch fully solves the original issue of this bug report (linux builts with qt4). Maybe the discussion of further improvements to qt5 builts in mxe-octave could be continued in a seperate bug report.

Hartmut <hardy>
Wed 26 Oct 2016 07:34:37 PM UTC, comment #27:

Does this problem also happen with Qt5?

Your patch seems fine to me if it fixes the problem for Qt4.

John W. Eaton <jwe>
Project Administrator
Sun 23 Oct 2016 09:33:08 PM UTC, comment #26:

I have compiled Octave with my proposed changes from comment #25 (change all QString::toLatin1 into QString::toLocal8Bit) and it does NOT help (with qt 4 under Linux). It seems that those encodings don't take effect "deep" enough in the GUI.

But I tried an even simpler patch, and it DOES help! If I re-add the line with "setCodecForCStrings" in octave-gui.cc for (only in qt4 builds), then all works fine:

  • running m-scripts in folders with Umlauts works fine from the GUI editor
  • using the inputdlg command to receive non-english characters, also works fine.

I will attach a tiny PATCH to show what I only did to solve all the observed behaviors reported in this bug report. I have tested this patch only for qt4 builds under Linux, so far. But I would nevertheless consider this patch very "low risk": It only changes the code for qt4 builts (qt5 builts will not change at all). And it only re-adds a single line of code, that was present in all former (GUI) releases as well.

If noone else finds any problems with this patch, then I would be happy to see it in the upcomaing Octave 4.2 release. It would make using the GUI a significant bit nicer for all non-english Octave users.

(file #38798)

Hartmut <hardy>
Sun 23 Oct 2016 01:01:45 PM UTC, comment #25:

I have searched the Qt documentation a bit. Those are my sources:

  • For our GUI text encoding before cset 2fd4f1a3f4a8 (works only in qt4):
  • For our GUI text encoding after cset 2fd4f1a3f4a8 (works in qt4 and qt5):
    • http://doc.qt.io/qt-5/qtextcodec.html#setCodecForLocale
    • http://doc.qt.io/qt-5/qstring.html#toLatin1
    • http://doc.qt.io/qt-5/qstring.html#toLocal8Bit

If I understand this correctly, then our former combination like this
was converting texts to a 8bit representation, according to the utf8 encoding.

Our new combination like this
is converting text to the latin1 8bit repesentation. This is not the same as the previous behavior. (If no locale is set, then this is a good replacement as mentioned in the qt4-qt5 transition document, but we had a locale set.)

Could the following be a better way to change the code to something that runs under qt4 and qt5?

I would technically be able to prepare a new cset for this, but I am hesitating to do such changes very deep in core Octave, because I have never used the qt libraries before. The files that might need a change like this, are Vt102Emulation.cpp, file-editor-tab.cc and workspace-model.cc.

Hartmut <hardy>
Sat 08 Oct 2016 02:23:24 PM UTC, comment #24:

Related to earlier posts. I did a complete clean and re-build of the MXE-octave version with Qt5 enabled. There are no weird segfaults due to having both qt4 and qt5 libraries present.

It doesn't appear to solve this issue because I can't generate an umlaut character in the Command Window. I can take an umlaut from another source, such as a web page or the notepad editor, and paste it into the Editor window. When I try the same thing with the Command window the umlaut character is removed and the rest of the text is pasted correctly.

Rik <rik5>
Project Administrator
Sat 01 Oct 2016 02:47:32 PM UTC, comment #23:

I'll have to let jwe take a look. I do see a few instances where toAscii has been replaced by toLatin1. Maybe that makes a difference.

Also, what strings are subject to UFT-8 may have changed.

Rik <rik5>
Project Administrator
Sat 01 Oct 2016 11:53:06 AM UTC, comment #22:

I've found the changeset that makes this happen:

http://hg.savannah.gnu.org/hgweb/octave/rev/2fd4f1a3f4a8
"allow building with Qt5 (bug #40252)" by jwe.

Using the repo version before this changeset makes both bugs (umlaut folders in GUI editor and inputdldg with umlauts) disapper in my Linux built (with qt4). Applying this single changeset makes both discussed bugs appear.

Hartmut <hardy>
Fri 30 Sep 2016 11:29:42 PM UTC, comment #21:

See the attached file 'myconfig'. It is written a tcsh script, but you can easily adapt it to bash or whatever.

For ccache, install that using your package manager. Once that is done, add the directory '/usr/lib/ccache' to the start of your path. The line from my .tcshrc file, for example, is

After that, initialize the cache size with something like this which sets a 10 Gigabyte cache.

Once that is done, go to the directory where you build, run 'myconfig' followed by 'make -jXX' where XX is the number of CPUS or sometimes CPUS+1.

You can verify that it is working by running

to see the cache statistics

(file #38634)

Rik <rik5>
Project Administrator
Fri 30 Sep 2016 09:58:51 PM UTC, comment #20:

My builds take more than one hour currently. Could you copy-paste me some sample commands of "configure" and "make" with the options you normally use to speed things up? Especially those Cxx flags, I have never used before. Neither pipe nor cchache. (I'm still during the first build process, I needed to restart because of bug #49039, I will now re-apply the corresponding patch each time.)

Hartmut <hardy>
Fri 30 Sep 2016 09:45:30 PM UTC, comment #19:

You might want to use configure to create a minimal octave build since you will be doing a full build quite frequently.

I would at least use '--disable-docs' to save some time. Other things I would personally add include '--disable-jit' and '--disable-java'.

The compilation stage will go faster if you don't use optimization so you might want to explicitly set CFLAGS and CXXFLAGS to have -O0. I also add '-pipe' so that you rely on memory for intermediate files rather than a slow disk.

I also use ccache which helps quite a lot. If I'm just re-building an incremental difference it usually only takes about 2 minutes with ccache.

Rik <rik5>
Project Administrator
Fri 30 Sep 2016 09:14:23 PM UTC, comment #18:

I have re-tested Octave-4.2.0-rc2 (from alpha.gnu.org, Qt4 if I understood this right) under Win7:

  • The original bug (running scripts in folders with umlaut) works FINE there.
  • Pantxo's inputdlg bug from comment #3 works FAILs.

So there really IS something to gain with changeing mxe Octave to Qt5. But we are now talking about two different bugs, to be precise.

I will start to "manually" bisect the Octave code between 4.0.3 and 4.2.0-rc2 (on my Ubuntu 14.04 machine with Qt4), and test for the two different mentioned bugs under Linux. This might take some days...

Hartmut <hardy>
Fri 30 Sep 2016 07:45:27 PM UTC, comment #17:

I tried "hg log -b default -l 10 -k encoding -M libgui/" but all of those are related to the editor or Windows specific.

Pantxo Diribarne <pantxo>
Project Member
Fri 30 Sep 2016 07:13:11 PM UTC, comment #16:

Are we talking about moving the mxe build to Qt5 prior to 4.2 release?
If so, considering the "build transparent" changes Qt brought between 4 and 5 (see e.g. bug #49053 on mac) and that only recently did we see Octave running with Qt5, I'd vote for accepting this regression and keep Qt4 stable build until 4.4 (or the 5??hours bisect but I am not proposing my self here :-)).

PS: I am very grateful to people who contributed the recent effort to be able to build Octave with Qt5.

Pantxo Diribarne <pantxo>
Project Member
Fri 30 Sep 2016 07:12:07 PM UTC, comment #15:

I dont' have any objections against moving to Qt5.

But I don't see how changing something with MXE Octave -which is ment to generate binaries for WINDOWS OS- would help for this bug. The current Windows installer (4.2.0-rc2 from alpha.gnu.org) seems to already be fine and doesn't even show this bug.

This regression bug was observed under LINUX, and changing MXE Octave wouldn't help with the Linux binaries, would it?

So even after moving MXE Octave to Qt5, this bug will persist with Linux binaries that are built with Qt4. (Linux binaries with Qt5 are fine, already now.)

I seem to be missing something here.

Maybe here is our difference in thinking:

  • Rik has observed (in comment #8) that the inputdlg test from commmnt #3 FAILS with an MXE-Octave and Qt4.
  • I have observed (in comment #7) that the original problem (GUI editor not running scripts in folders with umlaut) works FINE with the precompiled installer of MXE-Octave 4.2.0-rc2.

Is there a difference between the original bug (folder name) and the indputdlg test of commment #3? Is there a difference between Rik's self compiled MXE Octave and the "official" rc2 installer?

Hartmut <hardy>
Fri 30 Sep 2016 06:29:22 PM UTC, comment #14:

A bisection is a possibility, but it will be quite slow. There are a lot of changesets between 4.0.3 and 4.2.0-rc2 and the test is interactive so it can't be automated to have bisect run on its own. It's probably 5 hours of effort.

On the other hand, Qt4.8, the last version of the Qt4 branch, was released in 2011. It's worth moving to Qt5 in any case, and it would fix this bug.

Rik <rik5>
Project Administrator
Fri 30 Sep 2016 05:58:07 PM UTC, comment #13:

Since

  • my Octave 4.0.3 does not show this bug,
  • but Octave 4.2.0-rc2 does, and
  • both my Linux builts are built the same way (with qt4 when I understand this right)
  • and assuming that the qt4 libraries have not changed and not freshly introduced this bug

then I would like to ask: Isn't it very probably that we changed our Octave code somewhere between version 4.0.3 and 4.2.0-rc2 and introduced this behavior? Can we fix this by going back and look where it changed? Since it happens under Linux OS, and there is also a simple test script in comment #3, would a code bisection help here? (I don't have elaborated enough coding skills to do something like a bisection myself.)

Also, I thought this is a "Linux only" bug, at least does the official Windows installer 4.2.0-rc2 not show this behavior. So why would the introduction of a "qt5" option to MXE Octave be of any here in this case?

I do not want to be destructive or demotivating at all. I would just like to find a feasable way to remove this regression in the Octave Linux GUI with qt4. Or have I missed any relevant facts here?

Hartmut <hardy>
Fri 30 Sep 2016 10:10:48 AM UTC, comment #12:

Yeah, I would suggest a clean build if changing from qt4 to qt5 to ensure everything uses qt5.

John Donoghue <lostbard>
Project Member
Fri 30 Sep 2016 03:45:16 AM UTC, comment #11:

The Octave that was built installs and I'm able to run it. There are a few issues. When I try to switch to the Editor tab I get a segfault. Also, when I try to open the Preferences menu I get a segfault where it tells me that the following module is at fault.

So it seems that some Qt4 components are still being linked in. I suspect if they are all removed then it will work correctly.

Rik <rik5>
Project Administrator
Thu 29 Sep 2016 11:08:30 PM UTC, comment #10:

I'm trying a self-built MXE with the new Qt5 option. I'll let you know how things go in a few hours.

Rik <rik5>
Project Administrator
Thu 29 Sep 2016 07:25:07 PM UTC, comment #9:

I pushed an initial changeset to mxe-octave for building a default-octave target that builds qt5 if --enable-qt5 is specified in the configure options.

John Donoghue <lostbard>
Project Member
Wed 28 Sep 2016 05:38:50 AM UTC, comment #8:

I can confirm that a self-built version of Octave on Linux using Qt5 passes the inputdlg test in comment #3. A self-built MXE version fails on Windows, but it is built with Qt4. I rebuilt Octave on Linux using the configure option '--with-qt=4'. This version now fails.

I'm adding jwe and JohnD to the CC list for this bug. It seems like it would be useful to build the MXE version with Qt5.

Rik <rik5>
Project Administrator
Fri 23 Sep 2016 11:31:24 AM UTC, comment #7:

I do NOT observe this bug in the current Windows pre-release 4.2.0-rc2 (official exe-installer from alpha.gnu.org). Is this maybe built with Qt5?
But I still DO observe this bug, with my (self compiled on Ubuntu 14.04) version of Octave 4.2.0-rc2 under Linux.

Hartmut <hardy>
Tue 20 Sep 2016 07:16:58 AM UTC, comment #6:

@Dmitry: The Octave build from my original post was done with qt4, not with qt5. So this is compatible with your observations in comment #1.

Hartmut <hardy>
Tue 20 Sep 2016 06:57:36 AM UTC, comment #5:

This bug has several distinct differences to the older bug #42036:

  • This new bug happens on linux systems (and probably as well on Windows systems), but bug 42036 was only present on Windows systems.
  • This new bug was not yet present in Octave 4.0.3 (neither Linux nor Windows), but I guess 42036 was. So this new bug seems to me to be a regression.
  • This new bug only concerns the GUI elements of Octave, where as 42036 was also present in the command line box of the GUI, as well as in the CLI-version of Octave.
Hartmut <hardy>
Mon 19 Sep 2016 11:07:46 PM UTC, comment #4:

Is this the same bug as #42036?

Rik <rik5>
Project Administrator
Sat 17 Sep 2016 09:42:01 PM UTC, comment #3:

end of the previous post:

and the other way:

Pantxo Diribarne <pantxo>
Project Member
Sat 17 Sep 2016 09:39:50 PM UTC, comment #2:

I can confirm the bug and this used to work in 4.0.3.
The folowing example using qt inputdlg shows an encoding mismatch which may be a clue:

Pantxo Diribarne <pantxo>
Project Member
Sat 17 Sep 2016 09:32:59 PM UTC, comment #1:

openSUSE Tumbleweed: Qt5-based GUI works fine, Qt4-based GUI has problems with non-English characters in directory name.

Dmitry <rdmitry>
Sat 17 Sep 2016 08:57:00 PM UTC, original submission:

This happes on my Ubuntu 14.04 computer (with german language settings) with a self compiled version of roughly Octave-4.2.0-rc1 (more precisely with hg id b0caa61934fe)

Here is how to reproduce the behavior:

  • save a little m-file script in a folder with an umlaut in it. I created "/home/myself/Arbeitsfläche/test_umlaut.m". My little script just contains "a=1".
  • start Octave in GUI mode
  • (1) change the working directory to this very folder
  • start the m-file script from the command line, in my case by typing "test_umlaut"
    • This runs fine.
  • (2) change the working directory back to something different
  • open the same m-file script in the GUI editor, and run it from there. For example press F5 to run the script.
    • Now the GUI will NOT change the working directory properly, even when you answer the question "Would you like to change the working directory" with yes. Instead I see an error message (text see below).
  • (3) manually change the working directory to the proper folder.
  • run the file again from within the GUI editor, for example by pressing F5.
    • I see again an error message (saying "file or folder not found"):

The umlaut (ä) in the error message is also changed into a wrong character.

Note: Also the display in the top line of the GUI with the current folder name shows a wrong character for the umlaut.

In former versions like Octave 4.0.3 (also self compiled on the very same Ubuntu machine) this used to work fine. That's why I consider this to be a regression.

Hartmut <hardy>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

Attached Files
file #42167:  bug49118_win32_wchar_t_API_v9.patch added by mmuetzel (37KiB - application/octet-stream)
file #41670:  bug49118_win32_wchar_t_API_v8.patch added by mmuetzel (37KiB - application/octet-stream)
file #41401:  bug49118_win32_wchar_t_API_v7.patch added by mmuetzel (31KiB - application/octet-stream)
file #41251:  bug49118_win32_wchar_t_API_v6.patch added by mmuetzel (30KiB - application/octet-stream)
file #39829:  bug49118_win32_wchar_t_API_v5.patch added by mmuetzel (27KiB - application/octet-stream)
file #39620:  bug49118_win32_wchar_t_API_v4.patch added by mmuetzel (19KiB - application/octet-stream)
file #39166:  bug49118_win32_wchar_t_API_v3.patch added by mmuetzel (10KiB - application/octet-stream)
file #39058:  bug49118_win32_wchar_t_API_v2.patch added by mmuetzel (11KiB - application/octet-stream)
file #38634:  myconfig added by rik5 (234B - application/octet-stream)

 

Depends on the following items: None found

Digest:
   bug dependencies.

 

Carbon-Copy List
  • -unavailable- added by avinoam (Posted a comment)
  • -unavailable- added by mtmiller (Posted a comment)
  • -unavailable- added by mmuetzel
  • -unavailable- added by mmuetzel (Updated the item)
  • -unavailable- added by jwe (Posted a comment)
  • -unavailable- added by lostbard (Posted a comment)
  • -unavailable- added by rik5
  • -unavailable- added by rik5
  • -unavailable- added by pantxo (Posted a comment)
  • -unavailable- added by rdmitry (Posted a comment)
  • -unavailable- added by hardy (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only project members can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 25 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Mon 16 Oct 2017 06:15:04 PM UTCmmuetzelAttached File-=>Added bug49118_win32_wchar_t_API_v9.patch, #42167
    Sun 15 Oct 2017 07:36:41 PM UTCmmuetzelDependencies-=>bugs #52211 is dependent
    Sat 26 Aug 2017 08:18:30 PM UTCmmuetzelDependencies-=>bugs #51836 is dependent
    Sat 26 Aug 2017 07:48:42 PM UTCmmuetzelAttached File-=>Added bug49118_win32_wchar_t_API_v8.patch, #41670
    Mon 21 Aug 2017 03:45:07 PM UTCmmuetzelDependencies-=>bugs #40444 is dependent
    Thu 03 Aug 2017 02:04:50 PM UTCmmuetzelAttached File-=>Added bug49118_win32_wchar_t_API_v7.patch, #41401
      StatusConfirmed=>Patch Submitted
    Wed 19 Jul 2017 09:11:03 PM UTCmmuetzelAttached File-=>Added bug49118_win32_wchar_t_API_v6.patch, #41251
    Sat 25 Feb 2017 01:49:36 PM UTCmmuetzelAttached File-=>Added bug49118_win32_wchar_t_API_v5.patch, #39829
    Tue 31 Jan 2017 07:41:28 PM UTCrik5Carbon-CopyRemoved 72865=>-
    Mon 30 Jan 2017 10:40:46 PM UTCmmuetzelAttached File-=>Added bug49118_win32_wchar_t_API_v4.patch, #39620
    Tue 06 Dec 2016 08:15:45 PM UTCmmuetzelAttached File-=>Added bug49118_win32_wchar_t_API_v3.patch, #39166
      Item GroupRegression=>Incorrect Result
      Release4.2.0=>dev
      Operating SystemGNU/Linux=>Microsoft Windows
    Tue 06 Dec 2016 04:20:52 PM UTCmmuetzelDependencies-=>bugs #49768 is dependent
    Wed 23 Nov 2016 05:12:09 PM UTCmmuetzelAttached File-=>Added bug49118_win32_wchar_t_API_v2.patch, #39058
    Thu 17 Nov 2016 04:38:48 AM UTCmtmillerRelease4.2.0-rc1=>4.2.0
    Sun 13 Nov 2016 01:48:35 PM UTCmmuetzelAttached File-=>Added bug49118_win32_wchar_t_API.patch, #38945
      Carbon-Copy-=>Added tmacchant
    Mon 07 Nov 2016 05:31:16 PM UTCmmuetzelAttached File-=>Added bug49118_win32_wchar_chdir.patch, #38898
    Sun 23 Oct 2016 09:33:08 PM UTChardyAttached File-=>Added SetQt4EncodingForCStrings.patch, #38798
    Fri 30 Sep 2016 11:29:42 PM UTCrik5Attached File-=>Added myconfig, #38634
    Wed 28 Sep 2016 05:38:50 AM UTCrik5Carbon-Copy-=>Added jwe
      Carbon-Copy-=>Added lostbard

    Back to the top


    Powered by Savane 3.1-cleanup1