bugGNU Octave - Bugs: bug #55143, pkg install only succeeds on...

 
 

bug #55143: pkg install only succeeds on second attempt (Windows only)

Submitted by:  Hartmut <hardy>
Submitted on:  Sat 01 Dec 2018 10:17:32 AM UTC  
 
Category:  Octave Function Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Unexpected Error
Status:  Confirmed Assigned to:  None
Originator Name:  Open/Closed:  Open
Release:  dev Operating System:  Microsoft Windows

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

( Jump to the original submission)

Tue 18 Dec 2018 07:37:28 PM UTC, comment #6:

Please see also bug #45864, which is related, about simply running mkoctfile and trying to overwrite a loaded oct file. I had some initial ideas of a solution to try, but it didn't get enough testing from users on Windows to move it forward.

Mike Miller <mtmiller>
Project Administrator
Sat 01 Dec 2018 10:11:48 PM UTC, comment #5:

Thanks Hartmut for testing.

The issue is undoubtedly that in Windows, unlike Linux (Mac OSX I don't know), the binary module files of a package (the .oct files) are locked as long as their symbols are in the symbol table (or some other place in Octave where the "connections" to those functions are kept).
I suspect that binary functions get locked as soon as the subdir containing those functions is added to the path during loading of the package.

It just occurred to me that a plain "clear -f" might also wipe functions entered on the command line. So pkg.m needs to be able to find out exactly which functions have been loaded by a specific package in order to clear only those upon unloading the package.
That might complicate matters a little.

I had a brief look in pkg.m and .../<pkg>/private/unload_packages.m; my conclusion is that ATM I have no time to delve sufficiently deep into it. Moreover, I don't know the intricacies of loading binary OF package modules.

I'll add JWE to the cc list, I suppose he knows in detail how the mechanism of loading binary modules works. If he has a good hint I may pick up somewhere the next weeks, if no one else beats me to it.

Status -> Confirmed

Philip Nienhuis <philipnienhuis>
Project Member
Sat 01 Dec 2018 07:22:34 PM UTC, comment #4:

I've tested Philip's suggestion from comment #3 on Win7, and yes, it works fine just on the first go, then:

To recap: on Win7 we got unwanted behavior when I tried to upgrade a package that was already LOADED (I am pretty sure that I did not call any function from the package, yet.)

I've also tested the behavior under Linux (Ubuntu 18.04, Octave 4.4.1 release)

  • upgrade a (image) package that was currently unloaded
  • upgrade a (image) packagae that was currently loaded
  • upgrade a (image) package that was currently loaded and had already called a binary function from this package

I couldn't observe ANY unwanted behavior with Linux with all those ways to upgrade a package. So the Windows version of pkg seems to be much less tolerant in those circumstances compared to the Linux version.

Here is an example what works fine under Linux:

Hartmut <hardy>
Sat 01 Dec 2018 05:07:36 PM UTC, comment #3:

Just do:

- pkg load image
<run one of the binary image pkg modules>
- pkg unload image
- clear -f
- pkg uninstall image
- pkg install image-2.8.1.tar.gz

and see & report back what happens.
My guess is that it will work right away the first time.

FYI: for myself I'm pretty sure what your problem is. I usually encounter it when installing cross-builds made from a new mxe-octave build tree (I renew that every 2 months) and then finding out that when loading linear-algebra its old gsvd.oct shadows a core functions. (Like you I have OF packages loaded, in my case a.o., linear-algebra through .octaverc.)
To remove that gsvd.oct module I need to unload linear-algebra, run clear -f, and only then I can remove gsvd.oct in the linear-algebra binary subdir under ./lib/octave/. (To avoid this I have a patched linear-algebra w/o binary modules but I usually forget to adapt src/linear-algebra.mk in mxe-octave before building the installer).
It isn't hard to conceive that uninstalling any loaded OF package with binary modules needs the above-mentioned steps. It's is especially the "clear -f" that must not be forgotten, at least on Windows.

BTW This pkg.m behavior has existed in Octave for years and years, but I didn't bother as there's a workaround.
Now that there's a bug report it's definitely time to fix it. To do that some more info is needed:
- what happens on Linux when a loaded OF-package with binary modules is upgraded? could you try and find out, please?

Status -> still kept at "need info"
Item group -> Unexpected error

Philip Nienhuis <philipnienhuis>
Project Member
Sat 01 Dec 2018 03:04:43 PM UTC, comment #2:

Yes, the image package 2.8.0 WAS loaded before issuing the "pkg install" command the first time (because the load command is included in my startup.m script). At least for the majority of PCs where I did this.

I tried this with several Win7 PCs in a row (all of which I do not have access to any more). It might be that the same error also occured when NOT loading the image package before my first try with "pkg install". But I am not 100% sure about this.

You are right, I am also not 100% sure if this behavior was present with older Octave releases. I just have never encountered it before -> maybe no regression (but also maybe it is a regression)

If there are wishes what should be tested (on a Win7 machine) to further nail down this problem, then please let me know a detailed description of the necessary steps. (Because it takes quite a long time to fully un-install Octave, re-install Octave, install the image package...)

Hartmut <hardy>
Sat 01 Dec 2018 11:45:47 AM UTC, comment #1:

Was the 2.8.0 image package loaded before trying the first time?

I recognize your description of events for loaded packages with binary modules; the 'permission denied" error is thrown because binary modules in memory (+ probably symbol table) prevent removal of the pertinent .oct files.

Before "pkg install" of a package with binary models, the pkg command should first unload the package and then issue a "clear -f" (or maybe just clear the binary modules from the package in question, but AFAICS "clear -f" doesn't harm other binary modules).

I don't know if this is already implemented in pkg.m but doesn't work, or if it should be implemented. In case of the latter it doesn't seem to difficult to fix IMO.

Status => need info (just to be sure)
Release > dev
Item group - IMO should be "Unexpected error", not "Regression" but let's wait for Hartmut's confirmation

Philip Nienhuis <philipnienhuis>
Project Member
Sat 01 Dec 2018 10:17:32 AM UTC, original submission:

This happened on two different PCs with Win7:

  • I did a fresh install of Octave 4.4.1 release (64bit Win installer from website), this comes with image-2.8.0 package included.
  • I put the newest image package release zip-file into Octave's working directory (image version 2.8.1 from Forge website)
  • The (update) installation of the image package now throws an error message on the first attempt. But just repeating the command (without any changes at all) the second run then succeeds:

Is this a problem with the pkg command? I think this worked fine with older versions of Octave (also under Win7).

Hartmut <hardy>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by philipnienhuis
  • -email is unavailable- added by philipnienhuis
  • -email is unavailable- added by hardy (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can add your encouragement to it.
    This task has 0 encouragements so far.

    Only project members can vote.

     

     

     

    Follow 6 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2018-12-01 philipnienhuis StatusNeed Info => Confirmed
        Carbon-Copy- => Added jwe
    2018-12-01 philipnienhuis Carbon-Copy- => Added philipnienhuis
    2018-12-01 philipnienhuis Item GroupRegression => Unexpected Error
    2018-12-01 philipnienhuis StatusNone => Need Info
        Release4.4.1 => dev

    Back to the top


    Powered by Savane 3.3