bugGNU Octave - Bugs: bug #59923, mkoctfile fails on Windows if...

 
 

bug #59923: mkoctfile fails on Windows if installed in a path with spaces on a file system without short file names

Submitted by:  None
Submitted on:  Fri 22 Jan 2021 05:01:59 AM UTC  
 
Category:  Octave Function Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Unexpected Error or Warning
Status:  Fixed Assigned to:  None
Originator Name:  Originator Email:  -email is unavailable-
Open/Closed:  Closed Release:  dev
Operating System:  Microsoft Windows

Add a New Comment (Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

( Jump to the original submission)

Sun 07 Feb 2021 12:38:03 PM UTC, comment #19: 

Using double-quotes broke the builds with MXE Octave for gnu-linux.
I pushed an additional change that adds the surrounding double-quotes only when building for a mingw host:
https://hg.octave.org/mxe-octave/rev/eccf230ad6c1

Markus Mützel <mmuetzel>
Project Member
Tue 02 Feb 2021 06:51:37 AM UTC, comment #18: 

On jwe's buildbots, the patches to the scripts seem to apply successfully for the Octave targets for which they should. And they don't apply for the Octave targets for which they shouldn't.

No errors on compile time and in my runtime tests.

Closing as fixed.

Markus Mützel <mmuetzel>
Project Member
Sun 31 Jan 2021 06:02:19 PM UTC, comment #17: 

I pushed two patches (for post-install.bat and cmdshell.bat) that allow having spaces in the path where Octave is installed:
https://hg.octave.org/mxe-octave/rev/74d9b433846a
https://hg.octave.org/mxe-octave/rev/27b4f385c45d

Additionally, the following change removes the conversion to short paths for the default-octave target:
https://hg.octave.org/mxe-octave/rev/6805ab42c8aa

The changes in the latter patch can be reverted when we think that this is ready for the stable-octave and release-octave targets and the corresponding changes are applied to the script files directly. (That's probably not before Octave 7.)

We should probably follow up with new reports for any errors we'll find now with these changes. I already expanded the scope of this report probably more than I should have.

Markus Mützel <mmuetzel>
Project Member
Tue 26 Jan 2021 05:56:10 PM UTC, comment #16: 

I pushed the change that adds double-quotes around the paths for the default-octave target in MXE Octave here:
https://hg.octave.org/mxe-octave/rev/8645df9cd530

There are a few other spots where paths are converted to short paths in some scripts. I'll probably push a patch similar to the one in comment #15 soonish.

Marking as ready for test.

Markus Mützel <mmuetzel>
Project Member
Mon 25 Jan 2021 06:10:13 PM UTC, comment #15: 

The patch from comment #14 works for me.

But I'm having second thoughts whether it is ok to replace the double quotes in the `echo` part of the commands in subs-config-vals.in.sh by single quotes.
Is it possible that these strings will contain variables that need to be expanded? They are "plain" text for me at least.

I limited the number of strings that are modified to the ones which are used in mkoctfile and which are modified already anyway. I also added a FIXME note about this.

I pushed that part to default here:
https://hg.savannah.gnu.org/hgweb/octave/rev/78c660a8ba84

I'll probably push the patch in comment #14 to MXE Octave tomorrow or the day after if the buildbots are ok with the first change and no one opposes.

It might make sense to use the attached patch after that to have long file names in default Octave on Windows for developers and testers.

(file #50784)

Markus Mützel <mmuetzel>
Project Member
Mon 25 Jan 2021 11:48:52 AM UTC, comment #14: 

And sure enough, rebuilding everything from scratch did fail (on graphicsmagick fwiw).

I'm currently trying with the attached alternative patch in MXE Octave. It isn't pretty. But I hope it will do the trick.

(file #50781)

Markus Mützel <mmuetzel>
Project Member
Sun 24 Jan 2021 05:31:08 PM UTC, comment #13: 

I havnt tried the patch yet, but it wont surprise me if there are also some issues with quoting within some of the packages.

I vaguely recall there was also a saving of plots that has spaces in paths issues bug that I don't recall if was completely fixed.

John Donoghue <lostbard>
Project Member
Sun 24 Jan 2021 03:07:18 PM UTC, comment #12: 

@John: Thanks for that hint.
I did that, moved Octave to a folder with spaces in its name, and ran the test suite. It completed with 0 FAILs and no regressions.

So maybe there are not so many places that need to be changed after all.

With that change, I could reproduce the error from comment #0.

A few parameters that are used by mkoctfile as compiler and linker flags have paths that aren't surrounded by quotes.
Fixing that is surprisingly less straight forwards than I would have hoped.
There are basically two parts to that fix:
Part 1 is to allow using strings containing double quotes in the scripts that replace the respective parameters in the files in Octave.
Part 2 is adding double quotes around the respective paths in the configure variables in MXE Octave.

The issue I am seeing is that part 2 contains changes to Makefile.in in MXE Octave. Doing these changes without part 1 would break compilation of Octave in MXE Octave.

We maybe could apply part 1 to stable. But that would still leave the release target broken in MXE Octave.

I also didn't try to completely rebuild from scratch with MXE Octave. So it might be that other packages won't like these double quotes in their configure parameters either.

Anyway, with the two attached patches, I am able to use mkoctfile even if Octave is installed in a path containing spaces (and I remove the conversion to short file names from the start script).
Part 1: bug59923_mkoctfile_double_quotes.patch
Part 2: bug59923_mxe-octave_double_quotes.patch

(file #50778, file #50779)

Markus Mützel <mmuetzel>
Project Member
Sat 23 Jan 2021 09:07:49 PM UTC, comment #11: 

For testing mxe-octave without using 8.3 paths, it should be matter of just commenting out

OctavePath = fo.ShortPath

at around line 26 of the .vbs file

John Donoghue <lostbard>
Project Member
Sat 23 Jan 2021 02:55:25 PM UTC, comment #10: 

This isn't actually a regression. Changing item group.

Markus Mützel <mmuetzel>
Project Member
Fri 22 Jan 2021 10:19:30 AM UTC, comment #9: 

Ok. So the file system on your drive D: doesn't store short file names.

Installing Octave at the default destination in C: would probably work for you.
If you chose to install on D: (or any other file system that doesn't store short file names), you should select a destination folder that doesn't contain spaces as a work-around.

The real fix would be to not rely on short file names at all and correctly quote paths that might contain spaces where they are used in Octave.
This is a larger project and probably too much for a dot release.

As a first step, we should remove the conversion to short paths in the start up script imho because that masks potential issues on developer and tester machines (where conversion to short file names does work apparently).
But we should wait with that until after Octave 6.2.

In the mean time, users on affected machines should chose an installation folder that doesn't contain spaces.

Re-targeting to the development branch.

I believe there is a similar open bug report somewhere...

Markus Mützel <mmuetzel>
Project Member
Fri 22 Jan 2021 10:01:13 AM UTC, comment #8: 

system('dir /X D:\Program*')
 Volume in drive D has no label.
 Volume Serial Number is 686E-BE2A

 Directory of D:\

17/12/2020  02:07 PM               463              Program
22/01/2021  07:20 PM    <DIR>                       Program Files

05/06/2020  01:30 PM    <DIR>                       Program Files
 (x86)
               1 File(s)            463 bytes
               2 Dir(s)  1,594,719,330,304 bytes free
ans = 0

Anonymous
Fri 22 Jan 2021 09:19:07 AM UTC, comment #7: 

Aaah. I realized now that you installed Octave on drive D:.
Could you please test what the following returns in Octave?

system('dir /X D:\Program*')

Markus Mützel <mmuetzel>
Project Member
Fri 22 Jan 2021 08:55:19 AM UTC, comment #6: 

getenv('QT_PLUGIN_PATH')
ans = D:\Program Files\GNU Octave\Octave-6.1.0\mingw64\qt5\plugins

Anonymous
Fri 22 Jan 2021 08:46:10 AM UTC, comment #5: 

I'm using Windows 10 (not sure if you using Windows 8.1 is significant for this issue).

To check if the conversion to short file names in the vbs script file works on your system, could you please show what the following returns in Octave?

getenv('QT_PLUGIN_PATH')

Markus Mützel <mmuetzel>
Project Member
Fri 22 Jan 2021 07:55:56 AM UTC, comment #4: 

Windows 8.1

Start Octave using the default shortcut (GUI)

Target: %SYSTEMROOT%\system32\wscript.exe "D:\Program Files\GNU Octave\Octave-6.1.0\octave.vbs" --gui

Start in: %USERPROFILE%

I installed Octave to the default suggested for 6.1 which has moved from the default suggested path for earlier versions. The new default suggested has spaces the old did not.

Anonymous
Fri 22 Jan 2021 07:34:35 AM UTC, comment #3: 

Interesting. So the file system does store short file names for you. But still "OCTAVE_HOME" use the long file name (including spaces).

How do you start Octave? Which Windows version is this?

Markus Mützel <mmuetzel>
Project Member
Fri 22 Jan 2021 07:30:25 AM UTC, comment #2: 

version 6.1.0

OCTAVE_HOME
ans = D:\Program Files\GNU Octave\Octave-6.1.0\mingw64
>> system('dir /X C:\Program*')
 Volume in drive C is Windows
 Volume Serial Number is D06C-3F18

 Directory of C:\

17/09/2020  08:57 PM    <DIR>          PROGRA~1     Program Files

30/10/2020  10:21 AM    <DIR>          PROGRA~2     Program Files
 (x86)
               0 File(s)              0 bytes
               2 Dir(s)  16,644,935,680 bytes free
ans = 0

version 5.1.0

OCTAVE_HOME
ans = D:\Octave\Octave-5.1.0.0\mingw64
>> system('dir /X C:\Program*')
 Volume in drive C is Windows
 Volume Serial Number is D06C-3F18

 Directory of C:\

17/09/2020  08:57 PM    <DIR>          PROGRA~1     Program Files
30/10/2020  10:21 AM    <DIR>          PROGRA~2     Program Files (x8
6)
               0 File(s)              0 bytes
               2 Dir(s)  16,638,504,960 bytes free
ans = 0

version 4.2.1

OCTAVE_HOME
ans = D:\Octave\Octave-4.2.1
>> system('dir /X C:\Program*')
 Volume in drive C is Windows
 Volume Serial Number is D06C-3F18

 Directory of C:\

17/09/2020  08:57 PM    <DIR>          PROGRA~1     Program Files
30/10/2020  10:21 AM    <DIR>          PROGRA~2     Program Files (x8
6)
               0 File(s)              0 bytes
               2 Dir(s)  16,644,907,008 bytes free
ans = 0

Anonymous
Fri 22 Jan 2021 07:18:18 AM UTC, comment #1: 

This works for me.

But I'm assuming that this is caused by the fact that the file system on which Octave is installed for you doesn't save short file names.

Could you please show what the following commands return for you in Octave?

OCTAVE_HOME
system('dir /X C:\Program*')

On my system (German locale), I see:

>> OCTAVE_HOME
ans = C:\PROGRA~1\GNUOCT~1\OCTAVE~1.0\mingw64
>> system('dir /X C:\Program*')
 Volume in Laufwerk C: hat keine Bezeichnung.
 Volumeseriennummer: CB0E-80C0

 Verzeichnis von C:\

06.11.2020  20:14    <DIR>          PROGRA~1     Program Files
18.11.2020  11:26    <DIR>          PROGRA~2     Program Files (x86)
               0 Datei(en),              0 Bytes
               2 Verzeichnis(se), 151.263.674.368 Bytes frei
ans = 0

Markus Mützel <mmuetzel>
Project Member
Fri 22 Jan 2021 05:01:59 AM UTC, original submission:  

In the windows command prompt or within Octave typing

mkoctfile helloworld.cc

with helloworld.cc in the same directory containing the example from the documentation (or any other file)

#include <octave/oct.h>

DEFUN_DLD (helloworld, args, nargout,
           "Hello World Help String")
{
  octave_stdout << "Hello World has "
                << args.length () << " input arguments and "
                << nargout << " output arguments.\n";

  return octave_value_list ();
}

Results in the following error message

g++: error: Files\GNU: No such file or directory
g++: error: Octave\Octave-6.1.0\mingw64\/include: No such file or directory

I've tried placing the file in different directories including those which do not contain any spaces. The error message suggests this is a windows issue with forward and backslashes. The example has no issue compiling with version 5.1.0.

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by lostbard (Posted a comment)
  • -email is unavailable- added by mmuetzel (Posted a comment)
  •  

    Do you think this task is very important?
    If so, you can add your encouragement to it.
    This task has 0 encouragements so far.

    Only project members can vote.

     

     

     

    Follow 13 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2021-02-02 mmuetzel StatusReady For Test => Fixed
        Open/ClosedOpen => Closed
    2021-01-26 mmuetzel StatusIn Progress => Ready For Test
    2021-01-25 mmuetzel Attached File- => Added bug59923_mxe-octave_default_no_shortpath.patch, #50784
        StatusConfirmed => In Progress
    2021-01-25 mmuetzel Attached File- => Added bug59923_mxe-octave_double_quotes_v2.patch, #50781
    2021-01-24 mmuetzel Attached File- => Added bug59923_mkoctfile_double_quotes.patch, #50778
        Attached File- => Added bug59923_mxe-octave_double_quotes.patch, #50779
    2021-01-23 mmuetzel Item GroupRegression => Unexpected Error or Warning
    2021-01-22 mmuetzel StatusNeed Info => Confirmed
        Release6.1.0 => dev
        Summarymkoctfile helloworld.cc fails in 6.1.0 but is fine in 5.1.0 => mkoctfile fails on Windows if installed in a path with spaces on a file system without short file names
    2021-01-22 mmuetzel StatusNone => Need Info

    Back to the top


    Powered by Savane 3.7