bugGNU Octave - Bugs: bug #58225, Installing package fails when...

 
 

bug #58225: Installing package fails when package was already installed globally

Submitter:  John Donoghue <lostbard>
Submitted:  Wed 22 Apr 2020 12:31:24 PM UTC
   
 
Category:  Octave Function Severity:  3 - Normal
Priority:  5 - Normal Item Group:  None
Status:  Fixed Assigned to:  None
Originator Name:  lostbard Open/Closed:  * Closed
Release:  * 6.0.90 Operating System:  * Microsoft Windows
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Sun 24 May 2020 04:30:12 PM UTC, comment #27: 

Hmm, comment #25 (aimed at comment #9 and comment #10 here) should better fit at bug #58299, indeed.

AFAICS there's nothing more to do here after Markus grafted to stable in comment #17 and the ultimate doc fix.
Closing this bug report with "fixed"

Philip Nienhuis <philipnienhuis>
Group Member
Sun 24 May 2020 07:54:40 AM UTC, comment #26: 

@Philip: Not sure what you are trying to say with comment #25. In case that is about the default location for global and local packages: that kind of thing will be worked on in bug #58299.

Do you think that there are still open issues that need to be changed for this bug?

Markus Mützel <mmuetzel>
Group administrator
Fri 22 May 2020 08:09:22 PM UTC, comment #25: 

Further to comment #19, these days I have the following stanza in my ~/.octaverc to have separate local packages stores for different Octave releases:

vo = ver ("octave").Version(1);
pkg ("prefix", [ "~/octave" vo ], [ "~/octave" vo ]);
pkg ("local_list", [ "~/octave" vo "/.octave" vo "_packages" ]);


I tried this before but it didn't work at the time, but lately I found the culprit and fixed it in bug #58332.

Philip Nienhuis <philipnienhuis>
Group Member
Fri 22 May 2020 08:00:45 PM UTC, comment #24: 

Yeah, as I wrote it was a patch ("diff -u") rather than a cset.

You made a beautiful piece of documentation out of it, much better than my first throw, thank you very much :-)

Philip Nienhuis <philipnienhuis>
Group Member
Fri 22 May 2020 09:08:09 AM UTC, comment #23: 

@Philip: The patch that you submitted doesn't look like a Mercurial patch. Thus, it doesn't apply.

I made the changes manually, re-wrote part of it, fixed some errors and pushed the change to stable with your name here:
http://hg.savannah.gnu.org/hgweb/octave/rev/8b71a6cec9bf

Marking as ready for test.

Markus Mützel <mmuetzel>
Group administrator
Wed 06 May 2020 10:35:26 AM UTC, comment #22: 

Attached is a suggested patch for pkg.m's texinfo with some general information about installing packages.

I realize that most of it is already touched in the outline of commands further below, but IMHO the info there is of the type that is understood only if you already know about it. I hope the extra info in my patch helps to supply some coherent background for new users.


(file #49007)

Philip Nienhuis <philipnienhuis>
Group Member
Mon 04 May 2020 10:37:41 PM UTC, comment #21: 

Oopsie, sorry, bug #56775

Philip Nienhuis <philipnienhuis>
Group Member
Mon 04 May 2020 07:51:11 PM UTC, comment #20: 

@Philip: Bug #58225 is this very bug here. You probably meant to cite another bug.

AFAICT, the default value of global_install was unconditionally true on Windows before
http://hg.savannah.gnu.org/hgweb/octave/rev/baf16e6f498b on default
and http://hg.savannah.gnu.org/hgweb/octave/rev/ec77c790fce2 on  stable.
I don't see how it could get be set to false unless "pkg" was called with "-local".

It would be interesting to revisit the other bug to see what happened there.

Markus Mützel <mmuetzel>
Group administrator
Mon 04 May 2020 07:28:09 PM UTC, comment #19: 

@Markus comment #16;
I beg to differ; one of the reasons for bug #58225 was that pkg.m sometimes installed OF packages locally without me realizing it (BTW that is the bug report I mentioned in comment #2). Until that moment I figured, like you, that OF packages would by default always, consistently, be installed and/or updated in OCTAVE_HOME.
It may have been due to local circumstances which I'm unaware of then, but I've seen this happen on several Windows boxes the last year. Specifically I avoid the "-local" flag to avoid issues with several different Octave releases installed on my Windows boxes that would use the same local installations w/o guarantees for binary compatibility.

Philip Nienhuis <philipnienhuis>
Group Member
Mon 04 May 2020 10:36:59 AM UTC, comment #18: 

I opened bug #58299 about the issue from commment #9.

Markus Mützel <mmuetzel>
Group administrator
Mon 04 May 2020 09:23:59 AM UTC, comment #17: 

I grafted that change to stable here:
http://hg.savannah.gnu.org/hgweb/octave/rev/ec77c790fce2

This should resolve the original bug from comment #0.

I'm leaving the state as is because Philip wrote he'd like to improve the documentation here.

Markus Mützel <mmuetzel>
Group administrator
Mon 04 May 2020 08:14:41 AM UTC, comment #16: 

@Philip: "pkg" always tries to install in the global package store on Windows without that change (unless "-local" is set explicitly).
With that change, the behavior on Windows is similar to what "pkg" does on Linux: If Octave has elevated privileges (that would be running with root access on Linux), "pkg" uses the global store by default. Otherwise, it uses the local store by default.

Markus Mützel <mmuetzel>
Group administrator
Mon 04 May 2020 08:07:21 AM UTC, comment #15: 

Oh I got it the wrong way round, sorry.

But what would be the difference between testing with dev (7.0.0) and stable as regards this issue? after all apart from Markus' cset there's not that much functional difference in pkg.m between dev and stable, at least not s/th that would have influence here.

Philip Nienhuis <philipnienhuis>
Group Member
Mon 04 May 2020 08:03:06 AM UTC, comment #14: 

FTR, I do have that cset in my 70.0 build (my cross-builds usually have all kind of local mods and test patches) otherwise I couldn't even have tested __is_elevated_process__ .

Philip Nienhuis <philipnienhuis>
Group Member
Sun 03 May 2020 11:48:08 PM UTC, comment #13: 

We probally should if we are going to be installing to progran files.

John Donoghue <lostbard>
Group Member
Sun 03 May 2020 11:32:10 PM UTC, comment #12: 

@JohnD: You are right. This is only correctly handled on the default branch. And Philip and I seem to both have tested with that version.
Maybe we should graft that change to stable before the release:
http://hg.savannah.gnu.org/hgweb/octave/rev/baf16e6f498b

Markus Mützel <mmuetzel>
Group administrator
Sun 03 May 2020 10:44:35 PM UTC, comment #11: 

After looking at pkg.m, the issue was that I was using octave 6.0.1, which does not have the elevated checks in it.

John Donoghue <lostbard>
Group Member
Sun 03 May 2020 09:33:38 PM UTC, comment #10: 

For that matter even .m-file functions may not be compatible between Octave releases.

But your suggestion of having the local package .oct files live in separate subdirs for each Octave release is probably a good one IMO.

earlier on I've pondered a bit about this but I'm still a bit confused about how a new Octave release with a new api version could see the local packages already installed. I suppose a new Octave release would have to reinstall local packages to be able to have new api-compatible .oct files.

Philip Nienhuis <philipnienhuis>
Group Member
Sun 03 May 2020 07:36:39 PM UTC, comment #9: 

There might be another - possibly unrelated - issue:
The local package store is in %USERPROFILE%\octave. It is possible to install several different versions of Octave (very easy on Windows). All of these versions share the same local package store.
But .oct files are usually not binary compatible between versions.
So updating a local package (that includes binaries) in one version of Octave breaks it for the other version.

Maybe the local package store should also include the Octave version in the path (at least the major version).

Wasn't there a bug report about moving the local package store to %LOCALAPPDATA% or something? Does anyone remember which bug that was?
Or should I open a new bug report for this?

Markus Mützel <mmuetzel>
Group administrator
Sun 03 May 2020 06:43:33 PM UTC, comment #8: 

I can't reproduce it either with dev Octave installed in C:\Program Files.

__is_elevated_process__ consistently returns 0.

But I do see that the OF matgeom init script consistently fails at deleting verlessThan.m because of insufficient write permission in C:\Program Files.

Anyway I'll try to improve the Texinfo help to better outline the difference between local and global packages. e.g., ATM the description doesn't match what is actually happening.

Philip Nienhuis <philipnienhuis>
Group Member
Sun 03 May 2020 11:10:40 AM UTC, comment #7: 

@JohnD: I tried to replicate the issue but it seems to work correctly for me: The package installs locally shadowing the globally installed package.
The code never reaches install at line 211 if Octave runs without elevated privileges for me.
I used this command for the test with an installation that includes the packages bundled with MXE Octave:
pkg -forge install windows

What does "__is_elevated_process__" return for you? It's false for me.
Which Windows version did you try with? I'm on Windows 10 1910 64bit.

Markus Mützel <mmuetzel>
Group administrator
Tue 28 Apr 2020 09:31:31 AM UTC, comment #6: 

Indeed that is the obvious workaround.

Last weekend (while sailing) I was thinking over this a bit.

This is also going to hit unwary users when doing a "pkg upgrade".

Most users have little notion of the difference between local and global package installations. That subject is only briefly touched in pkg.m's help.
So what might be needed is (IMO):

  • a more general introduction in pkg.m's texinfo about what pkg.m does, and that, and why, there is a difference between local and global package installations
  • catching if Octave is run with or without admin credentials
  • when installing a package, find out if there is already a global installation of it
  • if yes:
    • if Octave runs with admin privs, install/upgrade the package right away
    • if Octave runs w/o, we have to choose [1]:
      • proceed silently with a local installation (catch the global uninstall process), knowing that a local installation shadows a global one. So we'd only need to insert the "-local" flag somewhere so that pkg. explicitly does a local install.
      • same, but emit a suitable warning.
      • emit an error and abort; explain that a "-local" flag is explicitly required.
  • if no:
    • leave the process as it is now (undefined; sometimes a global package is replaced, sometimes a newer package is installed locally)
    • depending on admin privs, install local or global.



Philip Nienhuis <philipnienhuis>
Group Member
Mon 27 Apr 2020 10:06:32 PM UTC, comment #5: 

It all works fine when i use the -local option, so it has to do with the fail over to local when permisions are not available.

John Donoghue <lostbard>
Group Member
Fri 24 Apr 2020 09:24:03 AM UTC, comment #4: 

OK, expect some results next week.

Philip Nienhuis <philipnienhuis>
Group Member
Thu 23 Apr 2020 07:06:42 AM UTC, comment #3: 

@Philip: That sounds good to me.
For point (2), there is probably a missing "ispc () && _is_elevated_process_ ()" check.

Markus Mützel <mmuetzel>
Group administrator
Wed 22 Apr 2020 05:55:57 PM UTC, comment #2: 

I once entered a (invalid) bug report where it turned out that locally installed packages in a local octave_packages file "shadow" global ones; I now remember I would add a cset to document this. (But ATM I can't find the bug report in Savannah's "my items".)

Anyway, based on this behavior it might be enough to just patch pkg.m to (1) warn that a global installed package will be shadowed by a newly to-be-installed package with the same name, and (2) avoid the "permission denied" error in such cases.

If you guys want I can look into it.

Philip Nienhuis <philipnienhuis>
Group Member
Wed 22 Apr 2020 04:05:11 PM UTC, comment #1: 

What is the behavior on Linux in this situation? I.e. if a package is already installed globally, and a user tries to update or re-install it without root permission (locally), does it fail or succeed installing?
We probably would like to achieve the same on Windows...

Markus Mützel <mmuetzel>
Group administrator
Wed 22 Apr 2020 12:31:24 PM UTC, original submission:  

Related I think to changing the installation path to program files (Bug #53124
), and checking to determine local vs global install (Bug #44548).

I have a package that was installed globally, and go to install an existing package (perhaps an updated version, but could be just reinstaling a new version)

pkg install somepackage


Recent changes detect that it cant install it globally and falls back to installing locally.

But as part of the pkg install it attempts to remove the old global) version


error: couldn't delete directory C:\Program Files\GNU Octave\Octave-6.0.1\mingw64\share\octave\packages\windows-1
.4.0: Permission denied
error: called from
    uninstall at line 127 column 11
    install at line 211 column 9
    pkg at line 513 column 9


Shouldn't I be able to install the package locally? Or if it shouldnt be allowed, error out because of the global package before going through all the time of compiling the package.

John Donoghue <lostbard>
Group Member

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #49007:  bug_58225_pkg.m_texinfo.patch added by philipnienhuis (3KiB - application/octet-stream)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by mmuetzel (Posted a comment)
  • -email is unavailable- added by lostbard (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 9 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2020-05-24 philipnienhuis StatusReady For Test Fixed
        Open/ClosedOpen Closed
    2020-05-22 mmuetzel StatusPatch Submitted Ready For Test
        Assigned tophilipnienhuis None
    2020-05-06 philipnienhuis StatusNone Patch Submitted
    2020-05-06 philipnienhuis Attached File- Added bug_58225_pkg.m_texinfo.patch, #49007
    2020-04-24 philipnienhuis Assigned toNone philipnienhuis
    2020-04-22 rik5 Carbon-CopyRemoved 72865 -
    2020-04-22 rik5 SummaryInstalling package fails when package was already instaledd globally Installing package fails when package was already installed globally

    Back to the top

    Powered by Savane 3.13-f8d8.
    Corresponding source code