bugGNU Octave - Bugs: bug #51632, make global pkg paths always...

 
 

bug #51632: make global pkg paths always relative to OCTAVE_HOME, no need to pkg rebuild -global when path changes

Submitted by:  Tasos Papastylianou <tpapastylianou>
Submitted on:  Mon 31 Jul 2017 06:57:31 PM UTC  
 
Category:  Octave Function Severity:  1 - Wish
Priority:  5 - Normal Item Group:  Feature Request
Status:  Fixed Assigned to:  Philip Nienhuis <philipnienhuis>
Originator Name:  Tasos Papastylianou <poacheR> Open/Closed:  Closed
Release:  dev Operating System:  Microsoft Windows

Add a New Comment (Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

( Jump to the original submission)

Sun 16 Feb 2020 11:32:54 PM UTC, comment #68: 

I use packages prefixes all the time.

- different octave versions
- different packages versions (mainly dev)
- my own order, e.g. ~/.local instead of my home folder

Juan Pablo Carbajal <juanpi>
Project Member
Sun 16 Feb 2020 10:20:13 PM UTC, comment #67: 

I think you'll need to write up something more detailed to describe what you mean by that, in a new bug report or in a blog post or something.

But "pkg prefix" has nothing to do with "global" packages, as I understand it. This bug report is about global packages, and that seems solved to me, and has nothing to do with prefixes.

So I'll take that as a yes and close as fixed.

Mike Miller <mtmiller>
Project Administrator
Sun 16 Feb 2020 09:48:02 PM UTC, comment #66: 

Still on my to-do list is extending this to work with prefixes as well.
The __OH__ "prefix" only works well for plain vanilla Octave installations where all OF packages live in OCTAVE_HOME.

That said, I wonder if there's a real-life use case for package prefixes. I can think up a few but whether they're really useful ...

Philip Nienhuis <philipnienhuis>
Project MemberIn charge of this item.
Sun 16 Feb 2020 08:18:12 PM UTC, comment #65: 

Has this bug report been fully resolved now? As I understand it, Octave 6 now tracks globally installed packages relative to OCTAVE_HOME automatically.

Mike Miller <mtmiller>
Project Administrator
Thu 21 Nov 2019 07:55:59 PM UTC, comment #64: 

I just tried the new octave_packages build thing (cherry-picked the two related csets into my old & overdue mxe-octave build tree) and it works beautifully.

Philip Nienhuis <philipnienhuis>
Project MemberIn charge of this item.
Thu 21 Nov 2019 06:12:51 PM UTC, comment #63: 

ok i added a check of the version number before doing pkg rebuild

John Donoghue <lostbard>
Project Member
Tue 19 Nov 2019 08:01:13 PM UTC, comment #62: 

I hadnt looked at it, but it probally could - I will take a look

John Donoghue <lostbard>
Project Member
Tue 19 Nov 2019 07:06:07 PM UTC, comment #61: 

@JohnD:
Thanks, another step to a more easily usable Octave (e.g., on thumb drives).
Can the "pkg rebuild" step during installation be made dependent on the Octave release? (just guessing: you probably looked at that.)
Otherwise it's no big deal, it takes only one or two seconds, much shorter than (what I guess is) rebuilding the font cache.

I've got a proof-of-concept for prefix/archprefix stuff (to make it all even more versatile) but finishing it will take a little bit longer.

Philip Nienhuis <philipnienhuis>
Project MemberIn charge of this item.
Tue 19 Nov 2019 01:04:03 PM UTC, comment #60: 

I added creation of the octave_packages file as part of the pkg-install.py command, and it seems to work ok for me when using the latest 6.0.0 sources.

https://hg.octave.org/mxe-octave/rev/4eae7db624e8

The mxe-installer file installer-files/post-install.bat still runs the pkg rebuild on install time for those that run the installer and will continue to ensure that the older 5.1 sources still end up with a usable pkg list (at least from the installer, or running the post install script)

When we move to the 6.0 versions, the rebuild can be removed from the post install script.

John Donoghue <lostbard>
Project Member
Sun 17 Nov 2019 08:02:50 PM UTC, comment #59: 

Ok so just some script running to build the final package structure would work.

The warning about octave was put in there as a couple of packages needed octave in order to install (not sure if is still needed). As was a little flakey due to paths/detections of things being different the linux build vs mxe build.

John Donoghue <lostbard>
Project Member
Sun 17 Nov 2019 05:11:39 PM UTC, comment #58: 

@Markus:
Thanks for testing & pushing - I didn't expect it to happen that soon :-)

@JohnD:
My current trick is copying over from my Windows partition / installed Octave by a rule in binary-dist-rules.mk
Today I upgraded my desktop box to Mageia 7, reinstalled all build tools and when running a fresh mxe-octave clone I saw:

[check requirements]
/bin/bash: octave: command not found
Warning - could not find native build version 6.0.0 of octave - some packages may fail, but continuing
:

....so the mxe-octave build tools are already prepared to invoke Octave on Linux in some way.
As most devs simply need a Linux build tree for Octave in the first place (to create the dist archive needed for mxe-octave) I figure a simple rule in Makefile.in could do the trick.

The octave_packages file is simply a text .mat file, ideally written by Octave itself.
One doesn't even need a current Octave when cross-building dev Octave; if one adds <mxe-octave>/.../mingw64/share/octave/6.0.0/m/pkg to the path the installed Octave (on Linux) will use the most recent scripts.
"pkg rebuild" is a pretty straightforward function that merely scans the package install directory (m-files) and adds the found package directories + related lib/octave/<arch>/ package directories + other info to the packages cellstruct. It doesn't run any of the functions so this is fairly safe.

Philip Nienhuis <philipnienhuis>
Project MemberIn charge of this item.
Sun 17 Nov 2019 03:32:51 PM UTC, comment #57: 

I did the following test with your first patch from comment #54:
Start Octave, call "pkg rebuild". "pkg list" now shows the paths including the current OCTAVE_HOME.
Close Octave, rename the folder where it is installed, re-start Octave. "pkg list" now shows the paths including the new OCTAVE_HOME. No "pkg rebuild" needed.

After some very minor clean-up, I pushed that patch here:
http://hg.savannah.gnu.org/hgweb/octave/rev/377f069841c1

Markus Mützel <mmuetzel>
Project Member
Sun 17 Nov 2019 01:19:39 PM UTC, comment #56: 

Taking a quick look, it should be pretty easy to add to the existing script, so I will take a look today or tommorrow unless someone else beats me to it

John Donoghue <lostbard>
Project Member
Sun 17 Nov 2019 12:25:53 PM UTC, comment #55: 

For the 'install' of packages in mxe cross build, it is currently running a python script tools/pkg-install.py which emulates a lot of the pkg install functionality.

Currently there is a function that does nothing in it (add_package_list) that intended to update the / create the packages list that could be filled in.

John Donoghue <lostbard>
Project Member
Sat 16 Nov 2019 11:19:16 PM UTC, comment #54: 

Here are two csets.

The first (bug_51632_3rd_fix_global_OCTAVE_HOME.cset) is the fix for the original issue, i.e.,

  • when writing the octave_packages file it replaces the OCTAVE_HOME part of packages installation paths with "__OH__"
  • when reading the octave_packages file it replaces "__OH__" with the then current OCTAVE_HOME contents.

This fix make Octave completely relocatable; I tried on Windows, but AFAICS it should also work on Linux.

The other cset (bug_51632_list_global_OCTAVE_HOME.cset) is for cosmetics, it replaces the OCTAVE_HOME part of package installation files with <OCTAVE_HOME> when running "pkg list", and is independent of the first cset. See below for an example.
Before:

>> pkg list
Package Name         | Version | Installation directory
---------------------+---------+-----------------------
     communications  |   1.2.1 | ...\mingw64\share\octave\packages\communications-1.2.1
            control *|   3.1.0 | ...\OCTAVE~1.0\mingw64\share\octave\packages\control-3.1.0
           database  |   2.4.3 | ...\OCTAVE~1.0\mingw64\share\octave\packages\database-2.4.3
:

After:

>> pkg list
Package Name         | Version | Installation directory
---------------------+---------+-----------------------
     communications  |   1.2.1 | <OCTAVE_HOME>\share\octave\packages\communications-1.2.1
            control *|   3.2.0 | <OCTAVE_HOME>\share\octave\packages\control-3.2.0
     data-smoothing  |   1.3.0 | <OCTAVE_HOME>\share\octave\packages\data-smoothing-1.3.0
:

The fix here (replacing just the OCTAVE_HOME parts) will work for the vast majority of the Octave installations. I've been busy with replacing the prefix/archprefix parts as well but that is a bit more complicated. Later on I'll come up with a fix that builds on top of the ones here.

I think the patches here are ready to be applied.

A related question is how to get an octave_packages file in mxe-octave. One way would be to have a local Octave installation on the build system (Linux) and use that with a script to generate the octave_packages file, along the lines of:

1. "pkg prefix" and "pkg archprefix" pointing to the octave installation in <mxe-octave>/dist/octave/mingw64

2. "pkg global_list" to specify where the octave_packages file needs to be created (<mxe-octave>/dist/octave/mingw64/share/octave)

3. "pkg rebuild" to actually create the octave-packages file.

(file #47875, file #47876)

Philip Nienhuis <philipnienhuis>
Project MemberIn charge of this item.
Fri 08 Nov 2019 06:52:16 PM UTC, comment #53: 

(Returning after a few distractions)

@Markus:
About 8.3 vs LFN path names comment #41:
I'm afraid no call to canonicalize-file-name can be dropped.
The 8.3 style file names are deep in the mxe-octave installer. See for example:

>> [a, b] = pkg ('prefix')
a = C:\Programs\Octave\OCTAVE~1.0_2\mingw64\share\octave\packages
b = C:\Programs\Octave\OCTAVE~1.0_2\mingw64\lib\octave\packages
>> OCTAVE_HOME
ans = C:\Programs\Octave\OCTAVE~1.0_2\mingw64

This feature request is about relocating unzipped mxe-octave installations. I think prebuilt packages in such installations are installed in OCTAVE_HOME anyway so prefixes would seem to be less relevant at first sight.
But to make the patch more general I think I'll make the pkg paths relative to the prefix and archprefix. In mxe-octave installers and zipdists it won't make any functional difference. The verbatim block above shows how OCTAVE_HOME is part of those prefixes in mxe-octave installations anyway.

A first final cset would be for Octave itself, I hope to come up with one this weekend.

The next step is patching mxe-octave to produce an octave_packages file in the installer and zipdist archive. Shouldn't be too hard but needs some sorting out; perhaps JohnD can help if I get stuck.

Philip Nienhuis <philipnienhuis>
Project MemberIn charge of this item.
Sun 03 Nov 2019 01:14:28 PM UTC, comment #52: 

Imho, it would probably be best to open a dedicated bug report for the short file names vs. long file names issue and not re-purpose one of the existing bugs.

I was referring to the "canonicalize_file_name" usages in the patch for this bug  here. But it might well be that some of the ones in the other patch can be dropped as well.

Markus Mützel <mmuetzel>
Project Member
Sun 03 Nov 2019 01:06:39 PM UTC, comment #51: 

Back on topic.

The patch in comment #36 is a bit unsafe in hindsight as it unconditionally prepends OCTAVE_HOME when reading the global packages list.

I have a (hopefully) better idea that can also incorporate other prefixes.

Philip Nienhuis <philipnienhuis>
Project MemberIn charge of this item.
Sun 03 Nov 2019 01:03:25 PM UTC, comment #50: 

Thanks for trying.

To stop thread drift (due to myself in the first place :-) ) shall we continue the discussion on e.g., bug #56267? (as that's where the fix was made that uncovered all the rest of the 8.3 vs. LFN issues.)
Or any other bug report that seems better suited.

Perhaps bug #57083, that has been closed but I'll reopen as soon as I find that indeed some canonicalize_file_name() calls can be dropped (comment #41 here).

Philip Nienhuis <philipnienhuis>
Project MemberIn charge of this item.
Sun 03 Nov 2019 10:29:13 AM UTC, comment #49: 

The changeset that introduced the short file names in MXE Octave is probably
https://hg.octave.org/mxe-octave/rev/a45374f7d9a3

There is no bug number in the commit message. So I don't know which specific error(s) should have been addressed by this change. And I can't test whether this is still necessary. I might not be with the changes to how we use the Windows file system API. Non-ASCII characters are ok now. If it was spaces in paths: they can be an issue on all platforms.

But to advance with the original issue: Don't worry about providing cross-builds. I am cross-building quite irregularly but quite frequently myself.
Could you please check which of the "canonicalize_file_name" calls can be omitted from your patch? Performance isn't really important for "pkg". But we try to avoid double checks in other places as well. See also comment #41.

Markus Mützel <mmuetzel>
Project Member
Sun 03 Nov 2019 10:00:55 AM UTC, comment #48: 

Here the log with Octave 5.1.0:

>> ver
----------------------------------------------------------------------
GNU Octave Version: 5.1.0 (hg id: d05d6eebde10)
GNU Octave License: GNU General Public License
Operating System: MINGW32_NT-6.2 Windows 6.2  x86_64
----------------------------------------------------------------------
Package Name         | Version | Installation directory
---------------------+---------+-----------------------
     communications  |   1.2.1 | D:\Octave\Octave-5.1.0.0\mingw64\share\octave\packages\communications-1.2.1
            control  |   3.1.0 | D:\Octave\Octave-5.1.0.0\mingw64\share\octave\packages\control-3.1.0
     data-smoothing  |   1.3.0 | D:\Octave\Octave-5.1.0.0\mingw64\share\octave\packages\data-smoothing-1.3.0
           database  |   2.4.3 | D:\Octave\Octave-5.1.0.0\mingw64\share\octave\packages\database-2.4.3
          dataframe  |   1.2.0 | D:\Octave\Octave-5.1.0.0\mingw64\share\octave\packages\dataframe-1.2.0
              dicom  |   0.2.1 | D:\Octave\Octave-5.1.0.0\mingw64\share\octave\packages\dicom-0.2.1
          financial  |   0.5.3 | D:\Octave\Octave-5.1.0.0\mingw64\share\octave\packages\financial-0.5.3
               fits  |   1.0.7 | D:\Octave\Octave-5.1.0.0\mingw64\share\octave\packages\fits-1.0.7
fuzzy-logic-toolkit  |   0.4.5 | D:\Octave\Octave-5.1.0.0\mingw64\share\octave\packages\fuzzy-logic-toolkit-0.4.5
                 ga  |  0.10.0 | D:\Octave\Octave-5.1.0.0\mingw64\share\octave\packages\ga-0.10.0
            general  |   2.1.0 | D:\Octave\Octave-5.1.0.0\mingw64\share\octave\packages\general-2.1.0
      generate_html  |   0.3.1 | D:\Octave\Octave-5.1.0.0\mingw64\share\octave\packages\generate_html-0.3.1
           geometry  |   3.0.0 | D:\Octave\Octave-5.1.0.0\mingw64\share\octave\packages\geometry-3.0.0
                gsl  |   2.1.1 | D:\Octave\Octave-5.1.0.0\mingw64\share\octave\packages\gsl-2.1.1
              image  |  2.10.0 | D:\Octave\Octave-5.1.0.0\mingw64\share\octave\packages\image-2.10.0
instrument-control  |   0.3.1 | D:\Octave\Octave-5.1.0.0\mingw64\share\octave\packages\instrument-control-0.3.1
           interval  |   3.2.0 | D:\Octave\Octave-5.1.0.0\mingw64\share\octave\packages\interval-3.2.0
                 io  |  2.4.12 | D:\Octave\Octave-5.1.0.0\mingw64\share\octave\packages\io-2.4.12
     linear-algebra  |   2.2.2 | D:\Octave\Octave-5.1.0.0\mingw64\share\octave\packages\linear-algebra-2.2.2
               lssa  |   0.1.3 | D:\Octave\Octave-5.1.0.0\mingw64\share\octave\packages\lssa-0.1.3
              ltfat  |   2.3.1 | D:\Octave\Octave-5.1.0.0\mingw64\share\octave\packages\ltfat-2.3.1
            mapping  |   1.2.1 | D:\Octave\Octave-5.1.0.0\mingw64\share\octave\packages\mapping-1.2.1
   mfile_basic_test  |   0.1.0 | C:\Users\Markus\AppData\Local\Temp\oct-G6lQc0\mfile_basic_test-0.1.0
mfile_minimal_test  |   0.1.0 | C:\Users\Markus\octave\mfile_minimal_test-0.1.0
      miscellaneous  |   1.2.1 | D:\Octave\Octave-5.1.0.0\mingw64\share\octave\packages\miscellaneous-1.2.1
                nan  |   3.1.4 | D:\Octave\Octave-5.1.0.0\mingw64\share\octave\packages\nan-3.1.4
             netcdf  |  1.0.12 | D:\Octave\Octave-5.1.0.0\mingw64\share\octave\packages\netcdf-1.0.12
              nurbs  |  1.3.13 | D:\Octave\Octave-5.1.0.0\mingw64\share\octave\packages\nurbs-1.3.13
                ocs  |   0.1.5 | D:\Octave\Octave-5.1.0.0\mingw64\share\octave\packages\ocs-0.1.5
             odepkg  |   0.8.5 | D:\Octave\Octave-5.1.0.0\mingw64\share\octave\packages\odepkg-0.8.5
              optim  |   1.5.3 | D:\Octave\Octave-5.1.0.0\mingw64\share\octave\packages\optim-1.5.3
        optiminterp  |   0.3.5 | D:\Octave\Octave-5.1.0.0\mingw64\share\octave\packages\optiminterp-0.3.5
         quaternion  |   2.4.0 | D:\Octave\Octave-5.1.0.0\mingw64\share\octave\packages\quaternion-2.4.0
           queueing  |   1.2.6 | D:\Octave\Octave-5.1.0.0\mingw64\share\octave\packages\queueing-1.2.6
             signal  |   1.4.0 | D:\Octave\Octave-5.1.0.0\mingw64\share\octave\packages\signal-1.4.0
            sockets  |   1.2.0 | D:\Octave\Octave-5.1.0.0\mingw64\share\octave\packages\sockets-1.2.0
          sparsersb  |   1.0.6 | D:\Octave\Octave-5.1.0.0\mingw64\share\octave\packages\sparsersb-1.0.6
            specfun  |   1.1.0 | D:\Octave\Octave-5.1.0.0\mingw64\share\octave\packages\specfun-1.1.0
            splines  |   1.3.2 | D:\Octave\Octave-5.1.0.0\mingw64\share\octave\packages\splines-1.3.2
         statistics  |   1.4.0 | D:\Octave\Octave-5.1.0.0\mingw64\share\octave\packages\statistics-1.4.0
                stk  |   2.5.1 | D:\Octave\Octave-5.1.0.0\mingw64\share\octave\packages\stk-2.5.1
            strings  |   1.2.0 | D:\Octave\Octave-5.1.0.0\mingw64\share\octave\packages\strings-1.2.0
             struct  |  1.0.15 | D:\Octave\Octave-5.1.0.0\mingw64\share\octave\packages\struct-1.0.15
             tisean  |   0.2.3 | D:\Octave\Octave-5.1.0.0\mingw64\share\octave\packages\tisean-0.2.3
                tsa  |   4.4.5 | D:\Octave\Octave-5.1.0.0\mingw64\share\octave\packages\tsa-4.4.5
              video  |   1.2.4 | D:\Octave\Octave-5.1.0.0\mingw64\share\octave\packages\video-1.2.4
            windows  |   1.3.0 | D:\Octave\Octave-5.1.0.0\mingw64\share\octave\packages\windows-1.3.0
             zeromq  |   1.3.0 | D:\Octave\Octave-5.1.0.0\mingw64\share\octave\packages\zeromq-1.3.0
>> OCTAVE_HOME
ans = D:\Octave\Octave-5.1.0.0\mingw64
>> diary off

So the conversion to 8.3 file names seems to be a no-op on that drive.

Markus Mützel <mmuetzel>
Project Member
Sun 03 Nov 2019 09:37:58 AM UTC, comment #47: 

OK, I got it wrong here, I stand corrected :-)
Thanks for the pointer.

Just curious: if you install Octave on that D:-drive (from the binary installer [*]) in a path with a fairly long file name, what does OCTAVE_HOME return?
I'm asking because AFAIK it's inside the NSIS installer where these 8.3 path names are created, and thus where much of this confusion comes from (the rest comes from pathseps). If that won't work on such an NTFS drive there may be another bug looming.

[*] If you want to try, a eperimental cross-build from Nov 2 is here:
https://prn183.stackstorage.com/s/n0MVtlEQ8J1e8yo
(up until cset 27615:83c1d4f75a36, but with local mods and incl. patch from comment #36)

Philip Nienhuis <philipnienhuis>
Project MemberIn charge of this item.
Sun 03 Nov 2019 07:11:02 AM UTC, comment #46: 
Markus Mützel <mmuetzel>
Project Member
Sun 03 Nov 2019 05:52:21 AM UTC, comment #45: 

@Philip: I don't know where you have that information from. But short file names don't work like you describe. See e.g. this Microsoft article about disabling them for a performance increase [1] - albeit with a compatibility warning for older programs.

As a final proof that I'm not hallucinating ;-)

D:\test>echo test > file_with_very_long_name.and_extension

D:\test>dir /x
Datenträger in Laufwerk D: ist DATA
Volumeseriennummer: A010-5609

Verzeichnis von D:\test

03.11.2019  06:47    <DIR>                       .
03.11.2019  06:47    <DIR>                       ..
03.11.2019  06:47                 7              file_with_very_long_name.and_extension
               1 Datei(en),              7 Bytes
               2 Verzeichnis(se), 83.087.142.912 Bytes frei

E:\test>echo test > file_with_very_long_name.and_extension

E:\test>dir /x
Datenträger in Laufwerk E: ist My Passport
Volumeseriennummer: 8470-964B

Verzeichnis von E:\test

03.11.2019  06:48    <DIR>                       .
03.11.2019  06:48    <DIR>                       ..
03.11.2019  06:48                 7 FILE_W~1.AND file_with_very_long_name.and_extension
               1 Datei(en),              7 Bytes
               2 Verzeichnis(se), 2.487.251.877.888 Bytes frei

"D:" is an internal drive, "E:" is an external USB drive. Note that the column that shows the short file name is empty on D:.

But you are right. This is off-topic.

[1]: https://support.microsoft.com/de-de/help/121007/how-to-disable-8-3-file-name-creation-on-ntfs-partitions

Markus Mützel <mmuetzel>
Project Member
Sat 02 Nov 2019 11:28:14 PM UTC, comment #44: 

BTW I see a bit of thread drift.

This bug / feature request is about making pkg.m deal with relocation, not so much LFN. Hopefully one of the core devs will pick up.

Philip Nienhuis <philipnienhuis>
Project MemberIn charge of this item.
Sat 02 Nov 2019 11:26:10 PM UTC, comment #43: 

Short file names don't work as you apparently think they do. They're not stored in the file system but used internally in user programs. NTFS or FAT32 makes no difference - internal Windows system calls are used to translate short file names to LFN and vice versa at or near the application level, not quite at the file system level.

If you'd install Octave in a subdir with a name with -say- 15 characters, you'd see that '__octave_config_info__ ("bindir")' echoes a short path name.
If you were to use a DOS program (even on Win10 supported through DOSBOX) you'll see that the DOS program also morphs LFN pathames into short (8.3 style) path names.

Short file names help a lot for MSYS / MSYS2 calls to cope with spaces in path names, that's why JohnD implemented them in the NSIS installer.

Philip Nienhuis <philipnienhuis>
Project MemberIn charge of this item.
Sat 02 Nov 2019 04:08:14 PM UTC, comment #42: 

Also on a more general note: I don't think that we should rely on the short file names to fix bugs at all. Support for short file names is optional for NTFS. Indeed, I had to attach an external NTFS volume for testing because the file systems on my internal volumes don't seem to store short file names.
Windows 10 came pre-installed. Maybe this is a new default - or some oddity of the manufacturer (ASUS).

Markus Mützel <mmuetzel>
Project Member
Sat 02 Nov 2019 03:42:00 PM UTC, comment #41: 

@Philip: Now that "path", "addpath", "rmpath" are fixed wrt short/long file names, can you please check whether you can omit some of the "canonicalize_file_name" calls in your patch?

Markus Mützel <mmuetzel>
Project Member
Thu 31 Oct 2019 06:08:59 PM UTC, comment #40: 

Markus, I think there's little wrong with your fix for bug #56267.

An issue is that all over the place in Octave we have a mix of "8.3" and LFN style paths.

One way to tackle that would be to deal with all 8.3 pathnames in one fell swoop, but that could defeat the very purpose they were created for.
The other tactic would be to just wait until more bugs pop up caused by mixing the path name styles on Windows. We'd need to keep track of where that is, as one fine day all those canonicalize_file_name () calls might be dropped.

My vote is for the latter option :-)

We can try to proactively grep for places where pathnames are compared. I suspect that number will be limited and I think we already got most of them.

Philip Nienhuis <philipnienhuis>
Project MemberIn charge of this item.
Thu 31 Oct 2019 12:57:14 PM UTC, comment #39: 

@Philip: Thanks for clearing this up. I'll try and come up with a better patch for bug #56267. Maybe over the weekend.

Markus Mützel <mmuetzel>
Project Member
Thu 31 Oct 2019 12:01:01 PM UTC, comment #38: 

@Markus:
The issue is no so much rmdir() but rather rmpath() that was handed 8.3 style pathnames but obviously couldn't match them with the LFN ones that appear now in the search path.
I see that on Windows, at various places in the code, 8.3 style pathnames still pop up, probably all originating from __octave_config_info__ .
In itself that's not so much a problem but since your cset 27497:5a0543de1e47 "Add canonical path when using addpath (bug #56267)", canonicalize_file_name calls are due in several places in the code that deal with paths.
A bit confusing is that all OF packages now have LFN pathnames in the path but core Octave functions still have 8.3 ones:

:
D:\Octave\Octave-6.0.0_20191031L\mingw64\share\octave\packages\io-2.4.13
D:\Octave\Octave-6.0.0_20191031L\mingw64\lib\octave\packages\io-2.4.13\x86_64-w64-mingw32-api-v53+
:
:
D:\Octave\OCTAVE~1.0_2\mingw64\lib\octave\6.0.0\site\oct\x86_64-w64-mingw32\
D:\Octave\OCTAVE~1.0_2\mingw64\lib\octave\site\oct\api-v53+\x86_64-w64-mingw32
:
:
D:\Octave\OCTAVE~1.0_2\mingw64\share\octave\6.0.0\m\general
:

Philip Nienhuis <philipnienhuis>
Project MemberIn charge of this item.
Thu 31 Oct 2019 08:04:15 AM UTC, comment #37: 

@Philip: If "rmdir" fails on 8.3 names, we should fix it.
Can you open a new bug report for this please?

Markus Mützel <mmuetzel>
Project Member
Wed 30 Oct 2019 09:54:44 PM UTC, comment #36: 

Was a little different, in fact rmpath() didn't recognize the 8.3 style path name it was handed by doc_cache_create, but another canonicalize-file_name() solved the issue.

Second cset attached, AFAICS this one works w/o errors on Windows and Linux for install, uninstall, load, unload, rebuild & describe.

(file #47769)

Philip Nienhuis <philipnienhuis>
Project MemberIn charge of this item.
Wed 30 Oct 2019 08:39:41 PM UTC, comment #35: 

Found the issue I mentioned in comment #34, it is rmpath() that apparently complains abut paths with backslash fileseps.
I'll have a look.

Philip Nienhuis <philipnienhuis>
Project MemberIn charge of this item.
Tue 29 Oct 2019 09:58:28 PM UTC, comment #34: 

Here's a first cset, aimed at replacing OCTAVE_HOME in the paths in the global octave-packages file.
It works mostly fine, but it's perhaps it's presently more a proof-of-concept.

It has been tested on Windows and Linux.
In Windows I ran Octave directly from the <mxe-octave>/dist/octave... installation, being able to use all prebuilt package still residing on the (read-only mounted) linux partition. To do that I simply copied the octave-packages file from Windows over to <mxe-octave>/dist/octave/mingw64/share/octave/ (did that in Linux where I can mount the Windows drive read-only) and back in Windows made a shortcut to the octave.vbs file in said mxe-octave installation tree.

So I think that proves that the global octave_packages is now relocatable.

I saw one glitch:
While trying to install the econometrics-1.1.2 package on Windows I ran into error messages about doc building. IIRC ?gen_lookfor_cache? or so? still used the 8.3 style paths and complained about some path or file not found - sorry I didn't save the messages (was late at night :-) )
In the end the search path contained entries for (IIRC) .../econometrics-1.1.2/doc and econometrics-1.1.2/packinfo
IIRC restarting Octave and a "pkg rebuild" solved it.

Comments welcome.

(file #47765)

Philip Nienhuis <philipnienhuis>
Project MemberIn charge of this item.
Mon 28 Oct 2019 07:47:42 PM UTC, comment #33: 

Brought here from bug #57083 (where I got to learn some internals of pkg.m) by a hint from Mike, and now that bug #57083 has been fixed, I think fixing the original bug here is in principle going to be fairly easy to do for both local and global packages.
It's actually along the lines that JWE sketched in comment #20.

It may be that paths relative to the various prefixes used by pkg.m work better than paths relative to OCTAVE_HOME and that is going to be a little more complicated as prefixes pop up at various places in pkg.m and its helper functions. Some investigation is needed.

As to the original bug, "pkg rebuild" will have to be run somewhere because AFAICS it's the only way an initial octave_packages file can be created. That "somewhere" will obviously be "sometime" in the process of mxe-octave, i.e., before the fairly complete Octave installation in <mxe-octave>/dist/ is being zipped or turned into an NSIS installer.

FTR, I think a fix can also be beneficial in some cases when running octave from read-only locations. But maybe some extra steps are needed for making docs available then.

I'll try to come up with a first provisional fix/cset and then we can see what else can be / needs to be done for the wishes in later comments.

Release -> dev

Philip Nienhuis <philipnienhuis>
Project MemberIn charge of this item.
Wed 09 Aug 2017 12:59:42 PM UTC, comment #32: 

I would be in support of that. Since OCTAVE_HOME is already set on launch it would make this a non issue.

Nicholas Jankowski <nrjank>
Tue 08 Aug 2017 09:34:03 PM UTC, comment #31: 

Since bug #51684 is the new meta-bug about all path portability issues, can we focus this bug on the specific issue that when OCTAVE_HOME changes, the user needs to run "pkg rebuild -global"?

I found an old thread where it was discussed to make the global pkg prefix no longer configurable and have it always derive from OCTAVE_HOME, avoiding the need to rebuild the cache when that path changes: https://lists.gnu.org/archive/html/octave-maintainers/2014-11/msg00018.html

I would propose that we revive that idea to resolve this bug, and make it so that the global cache only needs to be built once, or when a new pkg is installed, and changing OCTAVE_HOME when Octave moves to a different location is all that is needed.

Mike Miller <mtmiller>
Project Administrator
Tue 08 Aug 2017 01:59:36 AM UTC, comment #30: 

FYI, separate wishlist bug report created at bug #51684 for a truly portable" version.

Nicholas Jankowski <nrjank>
Thu 03 Aug 2017 08:29:27 PM UTC, comment #29: 

I agree that 'going truly portable' is probably worth a separate discussion elsewhere. 

the problem is either (1) actually let people know what to do, (2) remove the need for them to do anything.

(1) gets into the fact that the wiki is poorly linked from the main Octave and download pages. I see a link at the bottom of the main page, and a couple scattered Wiki links to the 'how to use' pages. Nothing to installation notes pages.

(2) Since the zip version's kind of a hack anyway, would the comment #17 proposal make this all a non-discussion? portability, Octave documentation/linking/wiki etc could be another issue for another day.

Alternatively, could mxe-octave add a line of text somewhere within the Octave welcome message (no idea what file that is) saying:
"You are using the Window zip package. On first use and anytime you change the Octave location you will need to run 'pkg rebuild' to access installed packages"

this will guarantee it's there for people to see.

Nicholas Jankowski <nrjank>
Thu 03 Aug 2017 06:34:58 PM UTC, comment #28: 

Doug makes an interesting point; the need for a pkg rebuild is less a 'at first launch' issue, but more of a 'whenever you change computer by treating the zip version as a portable octave' issue. Therefore it might make less sense to rely on the presence or absence of an `octave_packages` file, and more sense to simply check (via octaverc) whether `pkg list` is displaying appropriate output at each launch, and attempt a pkg rebuild if it is not, under the assumption that the paths have changed.

On the other hand, it would be reasonable to expect that a user that treats the octave zip-version "portably" in this way, is likely aware of the need to do a pkg rebuild, once they've done it once, and that this is a trivial thing to do so ... therefore to me this is still a documentation-first issue. It's an extremely simple issue to address, once you know what's going on. The biggest obstacle is not knowing what's going on in the first place, and not being aware of the relevant wiki post (as per the original stackoverflow post that triggered this discussion).

I feel the solution is more along the lines of having a simple line saying "you may need to rebuild / upgrade the packages", or a link to the wiki, next to the download link to the zipped version; this could also clarify what Mike is saying at the point of download, e.g. "link to alternative zip version: use this if you're looking for a standalone version that doesn't require admin rights, or to use as a 'portable' distribution (but see point on wiki regarding portability)". I think the fact that the download link in the main website simply redirects to a dry webpage of ftp links instead, devoid of any helpful instructions, is more relevant to the problem than an octaverc fix, however nice a touch the latter would be.

Tasos Papastylianou <tpapastylianou>
Thu 03 Aug 2017 06:11:23 PM UTC, comment #27: 

Yes, Portable is a separate issue and I guess we are getting sidetracked.  Doug's comment #26 describes a Portable distribution as it's usually intended. Octave is close to being so, but yes, currently Mike's comment #25 description of the zip usage is probably more correct.

What options are there for a 'run once' set of commands that would re-run after a 'reinstall' or 'upgrade' of the zip package? There is a

C:\Octave\octave-4.2.1\share\octave\site\m\once_only

folder.  What are the checks/conditions to execute things in that folder?

I know that on a new machine I get the 'this is your first time running Octave' dialogue since (i assume) it doesn't see anything in my User folder, but it only ever happens once on the machine. If I delete Octave and restore from the zip, or put a new version on, it doesn't create that dialogue again.

Nicholas Jankowski <nrjank>
Thu 03 Aug 2017 04:50:53 PM UTC, comment #26: 

Before I retired I had octave installed on a memory stick, and I carried it with me to different rooms. I then inserted the memory stick in whatever computer was in the room. I found that I had to do a rebuild so that the drive letter would work in that room. Each computer seemed to assign a different drive letter to the usb slot that I plugged the memory  stick into. I don't know if this is the same issue but it might be related, and I don't know how many people would still be doing it this way.

Doug Stewart <dastew>
Thu 03 Aug 2017 03:57:01 PM UTC, comment #25: 

To my mind, the zip distribution of Octave is an archive, that's all. Calling it a portable distribution is a bit disingenuous.

I haven't heard much call from users over the years for a truly portable version of Octave. If that's something that is now desired, maybe we should tackle the sub-issues in separate bug reports. For now, Octave definitely does need to know its own root directory (which can be overridden by the OCTAVE_HOME environment variable for many functions), and the pkg installation prefixes and cache file locations (which are not corrected to OCTAVE_HOME because they can be separately configured to be anywhere on the filesystem).

I thought the zip version was intended to be a workaround for not having admin privileges on a Windows system. Unpack the archive in a specific directory that the user does have privileges on and then run post-installation commands that a proper packaging format would normally run.

Mike Miller <mtmiller>
Project Administrator
Thu 03 Aug 2017 03:49:38 PM UTC, comment #24: 

Why would relative vs absolute paths even be an issue here?
All that would need to be done is to modify octaverc in the prepackaged windows zip version specifically such that it runs a simple pkg rebuild command at first launch (as per the one-liner below).
Are we getting sidetracked here, or is there a legitimate reason this would not be the preferred approach?

Also, regardless, it may still be worth treating this issue partly as a documentation issue, and simply make it more visible to a user at first launch; while any programmatic fix might address the `rebuild` issue, the user might still want to be informed that it may be worth to `update` their pkg database too, as per the wiki. And you certainly wouldn't want to automatically trigger an update programmatically at first launch.

Tasos Papastylianou <tpapastylianou>
Thu 03 Aug 2017 01:39:09 PM UTC, comment #23: 

re: comment #20:  that was my original thought, as I was messing with in comment #7 and comment #9. I had hoped that pkg would process the strings and expand out OCTAVE_HOME(), so that the only thing to do was store all the bundled packages with either relative path names or names relative to OCTAVE_HOME.  then no pkg rebuild would be required for either zip or exe. However, it appears I was mislead by how pkg displays the path and actually stores and uses the path, and haven't looked at it further.

re: comment #21: many people consider the zip versions of programs to be 'portable' versions. run from flash drive, etc. Octave zip is not actually a portable version since it relies on absolute paths and makes use of the user folder. Having everything able to work relative to OCTAVE_HOME, which the zip version successfully finds on startup, would be step 1 to making it actually portable in addition to solving the pkg rebuild issue. I guess 'relative to octave_home' is still an absolute path in a sense, depending on where the full path expansion occurs.

getting sidetracked into how to 'auto rebuild' as per comments #10-18 I stopped short of diving into pkg to see where it gets locked into absolute paths. First thought would be to see if there are places you could parse and 'eval' the string, but I know some people dislike eval in general.

jwe: (re: comment #19 re: comment #6)  hah... brilliant ... let's just pretend I never made that comment.

Nicholas Jankowski <nrjank>
Thu 03 Aug 2017 12:44:17 PM UTC, comment #22: 

for the building of octave_packages file,

cd $(HOST_PREFIX)
OCTAVE_HOME=. octave-cli --eval 'pkg rebuild -global'

appears to create the octave_packages file, with paths now as ./share/octave/packages/commuications-1.2.1 and ./lib/octave/packages/communications-1.2.1, so if the package database worked with names relative to OCTAVE_HOME, I think that would work.

John Donoghue <lostbard>
Project Member
Thu 03 Aug 2017 12:10:40 PM UTC, comment #21: 

Or, if there are actually legitimate uses for relative directory names (are there?) then we could extend the syntax of the file to somehow tag names that are relative to OCTAVE_HOME.  However, I'm not sure that we want to use $(OCTAVE_HOME) or similar since no other parts of Octave expand variables in path or directory names in this way.

John W. Eaton <jwe>
Project Administrator
Thu 03 Aug 2017 12:08:22 PM UTC, comment #20: 

It seems to me that we should just make the package database work with relative directory names so that any directory name that is stored in the file and that is not absolute is assumed to be relative to OCTAVE_HOME.

John W. Eaton <jwe>
Project Administrator
Thu 03 Aug 2017 12:05:46 PM UTC, comment #19: 

Nicholas: Regarding your statement in comment #6 that

cd %OCTAVE_HOME%

works, it only "works" because % is a comment character, so this expression is equivalent to calling "cd" without any arguments, which, in Octave, changes to the home directory.

John W. Eaton <jwe>
Project Administrator
Wed 02 Aug 2017 09:10:32 PM UTC, comment #18: 

I guess if there's a desire to remove those two lines from the octaverc after it's done could add:

  cd $(TOP_BUILD_DIR)/dist/share/octave/site/m/startup \
    && sed -i '$ d' octaverc \
    && sed -i '$ d' octaverc

after the zip line. (i'm sure there is a 'delete two lines' syntax for sed, but that's the limit of my knowledge)

Nicholas Jankowski <nrjank>
Wed 02 Aug 2017 09:00:43 PM UTC, comment #17: 

maintainers list discussion [1] advises trying to just make the change to the windows zip-distribution if possible.

I think this could be done changing the following in binary-dist-rules.mk in the 'define make-zip-dist' section (near line 194):

define make-zip-dist
  cd $(TOP_BUILD_DIR)/dist/share/octave/site/m/startup \
    && echo "## autogenerate the global octave_packages file for the Octave zipfile distro" >> octaverc  \
    && echo "if (exist (pkg ('global_list'), 'file') == 0), try, pkg rebuild, catch, end_try_catch, endif" >> octaverc

  echo "generating zip file..."
  cd $(TOP_BUILD_DIR)/dist \
    && zip -q -9 -r $(OCTAVE_DIST_NAME).zip $(OCTAVE_DIST_NAME)
endef

can anyone who actually knows how the makefile/building process works comment on whether that's right?

[1] http://octave.1599824.n4.nabble.com/any-impacts-of-adding-this-line-to-default-octaverc-tp4684287.html

Nicholas Jankowski <nrjank>
Wed 02 Aug 2017 06:17:10 PM UTC, comment #16: 

comment on the maintainers list mentioned that the code would throw an error every time Octave started if it was on a system where the user didn't have write access to the normal global_list location. Suggested using a try/catch to hide the error.

if (exist(pkg(&quot;global_list&quot;),&quot;file&quot;) == 0)
  try
    pkg rebuild;
  catch
  end_try_catch
endif

or the ultracompact 1 line version:


if (exist(pkg(&quot;global_list&quot;),&quot;file&quot;) == 0), try, pkg rebuild, catch, end_try_catch, endif

looks like this could just be added to the octaverc sitting in mxe-octave. But that would force this change on everyone to fix a problem only occurring in the Windows zip-distribution. Maybe a line could be added to the binary-dist-rules.mk file to append that line(s) to the octaverc file prior to zipping? Or maybe there's a better way to accomplish that?

Nicholas Jankowski <nrjank>
Tue 01 Aug 2017 08:31:20 PM UTC, comment #15: 

a couple options that would work:

if (isempty(file_in_path(fullfile(OCTAVE_HOME(), "share", "octave", "packages"), "octave_packages")))
  pkg rebuild;
endif

if (exist(pkg("global_list"),"file") == 0)
  pkg rebuild;
endif

I'm sure there are some others. the latter would cover it if that file designation ever changes.

Nicholas Jankowski <nrjank>
Tue 01 Aug 2017 08:21:37 PM UTC, comment #14: 

is there even a time running a pkg rebuild, when global octave_packages is missing, would break something?

Nicholas Jankowski <nrjank>
Tue 01 Aug 2017 08:19:46 PM UTC, comment #13: 

even better, you could check if the file referenced by pkg global_list exists; if not rebuild.

Tasos Papastylianou <tpapastylianou>
Tue 01 Aug 2017 08:13:21 PM UTC, comment #12: 

sorry, missed your comment #8, John.  I was intermittently changing  both dir and archprefix, mainly because I wasn't sure what archprefix did. 

Nicholas Jankowski <nrjank>
Tue 01 Aug 2017 07:58:34 PM UTC, comment #11: 

I had thought adding a simple find/replace on octave_packages before zipping would have been the simplest fix. Looks like not.

'on first launch' is still a bit of the problem.  When 4.3.0 comes out, I'm pretty sure it won't see it as first launch on my machine since I've run Octave before and my profile has the right files in place.  Same if I delete and 'reinstall'.  It'll only run 'first launch' the first time any version of Octave is run on the machine.   we want something that detects when this copy hasn't been run yet.

octave_packages is missing on first run. would putting a

if (~exist(fullfile(OCTAVE_HOME (),"path/to/octave_packages"))) pkg rebuild, endif;

or similar in .octaverc break anything?

Nicholas Jankowski <nrjank>
Tue 01 Aug 2017 07:18:06 PM UTC, comment #10: 

A simpler approach might be to just include the pkg rebuild command in the octaverc that gets executed on first launch. (I have checked this and it works, and it would be a simple bugfix). If there's a way to detect this is the first launch via some sort of flag, and only launch this once, even better; I've noticed there's a separate "octave-fistlaunch.vbs" script in the root folder: perhaps an environment variable can be set that is checked for within octaverc?

Whether there should be some form of documentation to inform of the potential need for a pkg update as well is a slightly separate issue.

Tasos Papastylianou <tpapastylianou>
Tue 01 Aug 2017 05:32:37 PM UTC, comment #9: 

ok. must be more to this. 

>> fullfile(OCTAVE_HOME (), "\share\octave\packages\signal-1.3.2")
ans = C:\Octave\OCTAVE~1.1\shareoctavepackagessignal-1.3.2

but put within the octave_packages, it produces:

Package Name  | Version | Installation directory
--------------+---------+-----------------------
      signal  |   1.3.2 | ...\share\octave\packages\signal

trying the long version of fullfile:

>> fullfile(OCTAVE_HOME(), "share", "octave", "packages", "signal-1.3.2")
ans = C:\Octave\OCTAVE~1.1\share\octave\packages\signal-1.3.2

but using that in octave_packages:

>> pkg list signal
Package Name  | Version | Installation directory
--------------+---------+-----------------------
      signal  |   1.3.2 | ...ile(OCTAVE_HOME(), "share", "octave", "packages",

it doesn't check for a valid path:

C:\Octave\OCTAVE~1.1\share\octave\packages123\signal-1.3.2

>> pkg list signal
Package Name  | Version | Installation directory
--------------+---------+-----------------------
      signal  |   1.3.2 | ...\OCTAVE~1.1\share\octave\packages123\signal-1.

>> pkgtest{1}.dir
ans = C:\Octave\OCTAVE~1.1\share\octave\packages123\signal-1.

but if I make it something arbitrarily short:

C:\Octave\OCTAVE~1.1\share\signal-1.3.2

>> pkg list signal
package signal is not installed.

so... haven't yet figured out how to be sure it's actually storing/reading the correct folder, and whether the truncation is just a display thing or if it's actually stored incorrectly.

so, not quite sure it'll work, and something about how octave_packages is read seems to 'sometimes' truncate things. eg:

>> pkgtest = pkg("list","signal");
>> pkgtest{1}.dir
ans = C:\Octave\OCTAVE~1.1\share\octave\packages\signal-1.3.2
>> pkgtest{1}.archprefix
ans = C:\Octave\OCTAVE~1.1\share\octave\packages\signal-1.3

despite the location being the same in the text file.

Nicholas Jankowski <nrjank>
Tue 01 Aug 2017 05:18:28 PM UTC, comment #8: 

I think you changed the wrong path - archprefix, instead of dir.

There is a comment #length that provides info on the length of the value, which has changed as you added the fullpath in.

John Donoghue <lostbard>
Project Member
Tue 01 Aug 2017 05:03:35 PM UTC, comment #7: 

interesting find:  octave_packages DOES work if you put a fullfile() function in for path.

replacing

C:\Octave\OCTAVE~1.1\lib\octave\packages\signal-1.3.2

with

fullfile(OCTAVE_HOME (), "\share\octave\packages\signal-1.3.2")

allows pkg list to read octave_packages.  however, it doesn't quite  work perfectly. 

>> pkg list signal
Package Name  | Version | Installation directory
--------------+---------+-----------------------
      signal  |   1.3.2 | ...\share\octave\packages\signal

not sure why it's truncating the -1.3.2 from the path.

fixing that, this should be a straightforward way to have the portable one find the packages right away taking advantage of the  environment variables set on octave startup to avoid needing to hardcode package links.

honestly, this would be an interesting way to store the global package paths anyway if we wanted to make a truly portable octave version.

Nicholas Jankowski <nrjank>
Tue 01 Aug 2017 02:20:11 PM UTC, comment #6: 

well, it's the syntax that normally works at the Octave interpreter  in windows (cd $OCTAVE_HOME gives an error, but cd %OCTAVE_HOME% works). Was hopeful it would be transparent to pkg, but no luck.  Would have made this a trivial fix. maybe add it to the ever longer 'pkg improvement wishlist' absent any other workaround...

Nicholas Jankowski <nrjank>
Tue 01 Aug 2017 02:07:31 PM UTC, comment #5: 

That's DOS environment syntax - $OCTAVE_HOME would be better, however pkg does not expand env variables

John Donoghue <lostbard>
Project Member
Tue 01 Aug 2017 01:47:36 PM UTC, comment #4: 

just to answer my own question: no, currently octave_packages cannot handle variables in paths.

Took my octave_packages file, replaced the absolute paths for the signal package from

C:\Octave\OCTAVE~1.1\lib\octave\packages\signal-1.3.2

to

%OCT_HOME%lib\octave\packages\signal-1.3.2

It renders the entire file unreadable to a 'pkg list'. i.e.:

pkg list:

>> pkg list
Package Name         | Version | Installation directory
---------------------+---------+-----------------------
     communications  |   1.2.1 | ...\share\octave\packages\communications-1.2.1
            control  |   3.0.0 | ...\share\octave\packages\control-3.0.0
     data-smoothing  |   1.3.0 | ...\share\octave\packages\data-smoothing-1.3.0
           database  |   2.4.2 | ...\share\octave\packages\database-2.4.2
          dataframe  |   1.1.0 | ...\share\octave\packages\dataframe-1.1.0
              dicom  |   0.2.0 | ...\share\octave\packages\dicom-0.2.0
          financial  |   0.5.0 | ...\share\octave\packages\financial-0.5.0
               fits  |   1.0.7 | ...\OCTAVE~1.1\share\octave\packages\fits-1.0.7
fuzzy-logic-toolkit  |   0.4.5 | ...\octave\packages\fuzzy-logic-toolkit-0.4.5
                 ga  |  0.10.0 | ...\OCTAVE~1.1\share\octave\packages\ga-0.10.0
            general  |   2.0.0 | ...\share\octave\packages\general-2.0.0
      generate_html  |   0.3.1 | ...\share\octave\packages\generate_html-0.3.1
           geometry  |   3.0.0 | ...\share\octave\packages\geometry-3.0.0
                gsl  |   2.0.0 | ...\OCTAVE~1.1\share\octave\packages\gsl-2.0.0
              image  |   2.6.1 | ...\share\octave\packages\image-2.6.1
instrument-control  |   0.2.3 | ...\octave\packages\instrument-control-0.2.3
           interval  |   2.1.0 | ...\share\octave\packages\interval-2.1.0
                 io  |   2.4.7 | ...\OCTAVE~1.1\share\octave\packages\io-2.4.7
     linear-algebra  |   2.2.2 | ...\share\octave\packages\linear-algebra-2.2.2
               lssa  |   0.1.2 | ...\OCTAVE~1.1\share\octave\packages\lssa-0.1.2
              ltfat  |   2.2.0 | ...\share\octave\packages\ltfat-2.2.0
            mapping  |   1.2.1 | ...\share\octave\packages\mapping-1.2.1
      miscellaneous  |   1.2.1 | ...\share\octave\packages\miscellaneous-1.2.1
                nan  |   3.1.2 | ...\OCTAVE~1.1\share\octave\packages\nan-3.1.2
             netcdf  |  1.0.11 | ...\share\octave\packages\netcdf-1.0.11
              nurbs  |  1.3.13 | ...\share\octave\packages\nurbs-1.3.13
                ocs  |   0.1.5 | ...\OCTAVE~1.1\share\octave\packages\ocs-0.1.5
             odepkg  |   0.8.5 | ...\share\octave\packages\odepkg-0.8.5
              optim  |   1.5.2 | ...\share\octave\packages\optim-1.5.2
         quaternion  |   2.4.0 | ...\share\octave\packages\quaternion-2.4.0
           queueing  |   1.2.5 | ...\share\octave\packages\queueing-1.2.5
             signal  |   1.3.2 | ...\share\octave\packages\signal-1.3.2
            sockets  |   1.2.0 | ...\share\octave\packages\sockets-1.2.0
          sparsersb  |   1.0.5 | ...\share\octave\packages\sparsersb-1.0.5
            specfun  |   1.1.0 | ...\share\octave\packages\specfun-1.1.0
            splines  |   1.3.2 | ...\share\octave\packages\splines-1.3.2
         statistics  |   1.3.0 | ...\share\octave\packages\statistics-1.3.0
                stk  |   2.4.2 | ...\OCTAVE~1.1\share\octave\packages\stk-2.4.2
            strings  |   1.2.0 | ...\share\octave\packages\strings-1.2.0
             struct  |  1.0.14 | ...\share\octave\packages\struct-1.0.14
           symbolic  |   2.6.0 | ...\share\octave\packages\symbolic-2.6.0
             tisean  |   0.2.3 | ...\share\octave\packages\tisean-0.2.3
                tsa  |   4.4.5 | ...\OCTAVE~1.1\share\octave\packages\tsa-4.4.5
              video  |   1.2.4 | ...\share\octave\packages\video-1.2.4
            windows  |   1.2.4 | ...\share\octave\packages\windows-1.2.4
             zeromq  |   1.2.1 | ...\share\octave\packages\zeromq-1.2.1

manually change paths in signal section of octave_packages to:

# name: dir
# type: sq_string
# elements: 1
# length: 55
%OCT_HOME%share\octave\packages\signal-1.3.2


# name: archprefix
# type: sq_string
# elements: 1
# length: 53
%OCT_HOME%lib\octave\packages\signal-1.3.2

and get:

>> pkg list
no packages installed.

change it back to "C:\Octave\Octave~1.1..." and 'pkg list' returns to normal.

so something about that syntax prevents pkg from parsing the entire file.  (also tested by changing the path to signal and going a 'pkg load control' and 'pkg load video', each listed before and after signal in the file, respectively. pkg returned a 'package control not installed' until I removed the %OCT_HOME% from signal.)

Nicholas Jankowski <nrjank>
Tue 01 Aug 2017 01:17:05 PM UTC, comment #3: 

Yes, but I guess what I meant was dp0 is used to grab the program location on startup and store it in an environment variable:

set OCT_HOME=%~dp0

 Could OCT_HOME be used within the installed octave packages file so that the rebuild command would not need to be run?

Nicholas Jankowski <nrjank>
Tue 01 Aug 2017 12:38:01 PM UTC, comment #2: 

~dp0 is used by the .bat file, which gets the real path before it is ever passed to octave, so currently octave doesn't know what ~dp0 is.

John Donoghue <lostbard>
Project Member
Mon 31 Jul 2017 08:59:13 PM UTC, comment #1: 

a good question:  do the paths 'need' to be hardcoded and absolute?  the zip version already uses ~dp0 to find the local path at startup.  could the zip packgage have a manually created package list with entries using that nomenclature? All subsequent pkg installs would get absolute paths, and so would a pkg rebuild. Would it work?

Nicholas Jankowski <nrjank>
Mon 31 Jul 2017 06:57:31 PM UTC, original submission:  

As per this StackOverflow post: https://stackoverflow.com/q/45399494/4183191

The need to run pkg rebuild (plus pkg update) in order for the preinstalled packages to become accessible in the zipped version, while documented somewhere on the wiki, it is highly unlikely that a new user would know this in advance, and is likely to be a source of frustration and "first papercut" to the very first experience of installing octave on windows in that way.

While there are probably good technical reasons to not provide a generic octave_packages file in the portable version (presumably the paths need to be hardcoded and absolute), this information would probably need to be documented in the README file, and / or in the information window that comes up at first launch, informing the new user of this problem, and / or redirecting to the wiki for further instructions at the very least.

Tasos Papastylianou <tpapastylianou>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files

 

Depends on the following items: None found

Digest:
   bug dependencies.

 

Carbon-Copy List
  • -email is unavailable- added by juanpi (Posted a comment)
  • -email is unavailable- added by mmuetzel (Posted a comment)
  • -email is unavailable- added by dastew (Posted a comment)
  • -email is unavailable- added by jwe (Posted a comment)
  • -email is unavailable- added by lostbard (Posted a comment)
  • -email is unavailable- added by nrjank (Posted a comment)
  • -email is unavailable- added by nrjank
  • -email is unavailable- added by tpapastylianou (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can add your encouragement to it.
    This task has 0 encouragements so far.

    Only project members can vote.

     

     

     

    Follow 18 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2020-02-16 mtmiller StatusReady For Test => Fixed
        Open/ClosedOpen => Closed
    2020-02-16 mtmiller StatusPatch Submitted => Ready For Test
    2019-11-16 philipnienhuis Attached File- => Added bug_51632_3rd_fix_global_OCTAVE_HOME.cset, #47875
        Attached File- => Added bug_51632_list_global_OCTAVE_HOME.cset, #47876
    2019-10-30 philipnienhuis Attached File- => Added bug_51632_2nd_fix_global_OCTAVE_HOME.cset, #47769
    2019-10-29 philipnienhuis Attached File- => Added bug_51632_first_fix_global_OCTAVE_HOME.cset, #47765
        StatusConfirmed => Patch Submitted
    2019-10-28 philipnienhuis Assigned toNone => philipnienhuis
        Release4.2.1 => dev
    2019-10-28 mtmiller Carbon-CopyRemoved 80942 => -
    2017-08-15 mtmiller CategoryConfiguration and Build System => Octave Function
        Severity3 - Normal => 1 - Wish
        Item GroupDocumentation => Feature Request
        StatusNone => Confirmed
        SummaryMS Windows portable (i.e. zip) version cannot find pre-installed packages, needs pkg rebuild, not mentioned at first run => make global pkg paths always relative to OCTAVE_HOME, no need to pkg rebuild -global when path changes
    2017-08-08 mtmiller Dependencies- => bugs #51684 is dependent
    2017-07-31 nrjank Carbon-Copy- => Added tmacchant@yahoo.co.jp

    Back to the top


    Powered by Savane 3.5