bugGNU Octave - Bugs: bug #58299, Local packages for different...

 
 

bug #58299: Local packages for different Octave versions cannot be installed in parallel

Submitted by:  Markus Mützel <mmuetzel>
Submitted on:  Mon 04 May 2020 10:35:40 AM UTC  
 
Category:  Interpreter Severity:  1 - Wish
Priority:  3 - Low Item Group:  Feature Request
Status:  Confirmed Assigned to:  None
Originator Name:  Open/Closed:  Open
Release:  6.0.90 Operating System:  Any

Add a New Comment (Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

Sun 24 May 2020 04:37:35 PM UTC, comment #5: 

In bug #58225  I offered a workaround, repeated here:

Put the following in e.g., your local .octaverc:

vo = ver ("octave").Version(1);
pkg ("prefix", [ "~/octave" vo ], [ "~/octave" vo ]);
pkg ("local_list", [ "~/octave" vo "/.octave" vo "_packages" ]);

which leads to different local package stores for different versions of Octave.
It could even be incorporated in the global (site-wide) .octaverc

Philip Nienhuis <philipnienhuis>
Project Member
Tue 05 May 2020 08:39:44 PM UTC, comment #4: 

@Markus: see bug #36477, bug #40444, bug #57463 and bug #55822

To be precise in bug #58225 I referred to .m files in OF packages, and it's not those .m-files that differ between Octave versions but rather many OF packages have .m files that explicitly depend on some range of Octave versions. (sorry for nitpicking)
FWIW my experience is that .m-files usually have a much longer lifetime across Octave releases than .oct files. Other than that I think Mike's suggestion is a good one.

But maybe the mxe-octave installer should be adapted to wipe local package directories relating to its Octave version (after user consent, of course).

This bug report is about locally installed packages but for global packages I see a lot of similarities.
On Windows all global stores are installed separately for mxe-octave cross-builds so little problems there. But AIU on *nix the global package stores (both in share/ and lib/) are shared between Octave versions. Will e.g., updating or reinstalling a global package running a newer Octave version (with different api no.) lead to removal of the existing subdir with .oct files for the older api? (I guess so, looking at the logic in pkg.m)

Philip Nienhuis <philipnienhuis>
Project Member
Mon 04 May 2020 09:42:52 PM UTC, comment #3: 

Ok, confirmed. I've found that by deleting the package cache file, Octave doesn't know about the other versions of the package oct files and won't delete them, but then fails when it tries to extract doc strings from the incompatible oct files. Added that fact to bug #57826.

So yes, I think we're on the same page on all operating systems, and I agree it would be a good enhancement to install packages by default in a per-Octave-version directory in the user's home / profile directory. I think it's an enhancement because most users don't regularly use multiple versions of Octave at the same time, and normally need to reinstall all packages when updating to a new version anyway.

IMHO, that directory should be based on the major version of Octave, so packages installed in version 7.1.0 are still found and work in version 7.3.0.

Mike Miller <mtmiller>
Project Administrator
Mon 04 May 2020 08:46:19 PM UTC, comment #2: 

I tried by installing the "windows" package locally with the following command in Octave 5.2:

pkg install -local -forge windows

After this, the package is installed in C:\Users\Markus\octave\windows-1.4.0:

>> cd C:\Users\Markus\octave\windows-1.4.0
>> dir
.                           @octave_com_object          iscom.m                     packinfo                    x86_64-w64-mingw32-api-v53
..                          doc-cache                   mat2xls.m                   test

Then I closed Octave 5.2 and tried to install the package again using the same command in a self-compiled Octave 6.0.1.
The installation succeeds. But all files from the previous installation are gone. In particular the "x86_64-w64-mingw32-api-v53" with the binaries for Octave 5.2 has disappeared:

>> cd C:\Users\Markus\octave\windows-1.4.0
>> dir

.                           ..                          @octave_com_object          doc-cache                   iscom.m                     mat2xls.m                   packinfo                    test                        x86_64-w64-mingw32-api-v54

This might lead to problems if a user wants to change between different Octave versions frequently and has local packages.

Markus Mützel <mmuetzel>
Project Member
Mon 04 May 2020 08:03:01 PM UTC, comment #1: 

Putting aside the issue of modifying the m-files dynamically, can you please test whether this works or doesn't work on Windows? It would be best if we can start from a known baseline of what does work and what would be nice to enhance.

On GNU/Linux, with the default configuration, the same set of local packages can be installed and used with multiple Octave versions. M-files are installed on top of each other in the same directory, so again, assume that's correct for now. Oct files are installed in a directory that is named for the CPU, OS, and Octave API version, so multiple sets of oct files can coexist. The local package store is shared between versions, but that is not a bug, again assuming the same set of packages.

There is definitely room for improvement, but please confirm first that you can get the same current behavior on Windows so we characterize this correctly.

I've offered this before elsewhere, but on my system I install packages in $XDG_DATA_HOME/octave/$OCTAVE_VERSION_MAJOR/packages (e.g. ~/.local/share/octave/6/packages/...).

Mike Miller <mtmiller>
Project Administrator
Mon 04 May 2020 10:35:40 AM UTC, original submission:  

The local package store for Windows is in %USERPROFILE%\octave atm.
It is possible to install several different versions of Octave (very easy on Windows). All of these versions share the same local package store.
But .oct files are usually not binary compatible between versions. In another bug report [1], Philip wrote that also some .m files might be different depending on the Octave version.
So updating a local package in one version of Octave can break it for the other version.

This issue will get more prominent because we plan on installing packages to the local store by default on Windows starting Octave 6.

Maybe the local package store should also include the Octave (major) version or API version in the path.

I believe there was a bug report about moving the local package store to %LOCALAPPDATA% or something. But I can no longer find that bug. So opening a new bug here.

I'd like to propose to store the local packages at %LOCALAPPDATA%/octave/{version} where {version} is the string returned by "version" (e.g. "6.1.0").

While this bug is focused mainly on the situation on Windows, it might make sense to make a similar change on all platforms. After all, after updating to a newer Octave version (or a downgrade fwiw), it is necessary to recompile all installed packages. So it would make sense to not list them from their old location when they probably won't work anyway.

Any thoughts?

[1]: https://savannah.gnu.org/bugs/index.php?58225#comment10

Markus Mützel <mmuetzel>
Project Member

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by mmuetzel (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can add your encouragement to it.
    This task has 0 encouragements so far.

    Only project members can vote.

     

     

     

    Follow 4 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2020-05-04 mtmiller Severity3 - Normal => 1 - Wish
        Priority5 - Normal => 3 - Low
        Item GroupUnexpected Error => Feature Request
        StatusNone => Confirmed

    Back to the top


    Powered by Savane 3.5