bugGNU Octave - Bugs: bug #55143, pkg install only succeeds on...

 
 

bug #55143: pkg install only succeeds on second attempt (Windows only)

Submitter:  Hartmut <hardy>
Submitted:  Sat 01 Dec 2018 10:17:32 AM UTC
   
 
Category:  Octave Function Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Unexpected Error or Warning
Status:  Confirmed Assigned to:  None
Originator Name:  Open/Closed:  * Open
Release:  * dev Operating System:  * Microsoft Windows
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Wed 19 Dec 2018 07:19:48 PM UTC, comment #10: 

OK Mike, makes sense, we'll leave it open for a while.



Philip Nienhuis <philipnienhuis>
Group Member
Wed 19 Dec 2018 05:41:23 PM UTC, comment #9: 

I don't think it's an exact duplicate, because pkg.m may need some additional code even after bug #45864 is fixed. That bug is solely about mkoctfile, this bug is about removing directories containing oct files.

 Mike Miller <mtmiller>
Group Member
Wed 19 Dec 2018 12:55:32 PM UTC, comment #8: 

I have no clue if this bug is a duplicate of bug #45864, but I will trust you on this one :) So for me it would be fine to

  • close this bug report, and
  • mark bug #45864 as some kind of "bug" with "normal" priority (and not just a "wish" and "feature request" with priority "low", as it is currently)


If you need further Windows testing to make progress with bug #45864, then just let me know, there.

 Hartmut <hardy>
Wed 19 Dec 2018 11:04:47 AM UTC, comment #7: 

@Hartmut,
I want to close this bug as it is clearly a duplicate of bug #45864. Would you agree?

 Philip Nienhuis <philipnienhuis>
Group Member
Tue 18 Dec 2018 07:37:28 PM UTC, comment #6: 

Please see also bug #45864, which is related, about simply running mkoctfile and trying to overwrite a loaded oct file. I had some initial ideas of a solution to try, but it didn't get enough testing from users on Windows to move it forward.

 Mike Miller <mtmiller>
Group Member
Sat 01 Dec 2018 10:11:48 PM UTC, comment #5: 

Thanks Hartmut for testing.

The issue is undoubtedly that in Windows, unlike Linux (Mac OSX I don't know), the binary module files of a package (the .oct files) are locked as long as their symbols are in the symbol table (or some other place in Octave where the "connections" to those functions are kept).
I suspect that binary functions get locked as soon as the subdir containing those functions is added to the path during loading of the package.

It just occurred to me that a plain "clear -f" might also wipe functions entered on the command line. So pkg.m needs to be able to find out exactly which functions have been loaded by a specific package in order to clear only those upon unloading the package.
That might complicate matters a little.

I had a brief look in pkg.m and .../<pkg>/private/unload_packages.m; my conclusion is that ATM I have no time to delve sufficiently deep into it. Moreover, I don't know the intricacies of loading binary OF package modules.

I'll add JWE to the cc list, I suppose he knows in detail how the mechanism of loading binary modules works. If he has a good hint I may pick up somewhere the next weeks, if no one else beats me to it.

Status -> Confirmed

Philip Nienhuis <philipnienhuis>
Group Member
Sat 01 Dec 2018 07:22:34 PM UTC, comment #4: 

I've tested Philip's suggestion from comment #3 on Win7, and yes, it works fine just on the first go, then:


>> pkg load image %ver 2.8.1
>> imdilate(eye(3),ones(2))
ans =

   1   1   0
   1   1   1
   0   1   1

>> pkg unload image
>> clear -f
>> pkg uninstall image
>> pkg install image-2.8.1.tar.gz
For information about changes from previous versions of the image package, run 'news image'.
>>


To recap: on Win7 we got unwanted behavior when I tried to upgrade a package that was already LOADED (I am pretty sure that I did not call any function from the package, yet.)

I've also tested the behavior under Linux (Ubuntu 18.04, Octave 4.4.1 release)

  • upgrade a (image) package that was currently unloaded
  • upgrade a (image) packagae that was currently loaded
  • upgrade a (image) package that was currently loaded and had already called a binary function from this package


I couldn't observe ANY unwanted behavior with Linux with all those ways to upgrade a package. So the Windows version of pkg seems to be much less tolerant in those circumstances compared to the Linux version.

Here is an example what works fine under Linux:


>> pkg load image
>> pkg list
Package Name  | Version | Installation directory
--------------+---------+-----------------------
       image *|   2.8.0 | /home/hartmut/octave/image-2.8.0
>> imdilate(eye(3),ones(2));
>> pkg install image-2.8.1.tar.gz
For information about changes from previous versions of the image package, run 'news image'.
>>



 Hartmut <hardy>
Sat 01 Dec 2018 05:07:36 PM UTC, comment #3: 

Just do:

- pkg load image
  <run one of the binary image pkg modules>
- pkg unload image
- clear -f
- pkg uninstall image
- pkg install image-2.8.1.tar.gz

and see & report back what happens.
My guess is that it will work right away the first time.

FYI: for myself I'm pretty sure what your problem is. I usually encounter it when installing cross-builds made from a new mxe-octave build tree (I renew that every 2 months) and then finding out that when loading linear-algebra its old gsvd.oct shadows a core functions. (Like you I have OF packages loaded, in my case a.o., linear-algebra through .octaverc.)
To remove that gsvd.oct module I need to unload linear-algebra, run clear -f, and only then I can remove gsvd.oct in the linear-algebra binary subdir under ./lib/octave/. (To avoid this I have a patched linear-algebra w/o binary modules but I usually forget to adapt src/linear-algebra.mk in mxe-octave before building the installer).
It isn't hard to conceive that uninstalling any loaded OF package with binary modules needs the above-mentioned steps. It's is especially the "clear -f" that must not be forgotten, at least on Windows.

BTW  This pkg.m behavior has existed in Octave for years and years, but I didn't bother as there's a workaround.
Now that there's a bug report it's definitely time to fix it. To do that some more info is needed:
- what happens on Linux when a loaded OF-package with binary modules is upgraded? could you try and find out, please?

Status -> still kept at "need info"
Item group -> Unexpected error

Philip Nienhuis <philipnienhuis>
Group Member
Sat 01 Dec 2018 03:04:43 PM UTC, comment #2: 

Yes, the image package 2.8.0 WAS loaded before issuing the "pkg install" command the first time (because the load command is included in my startup.m script). At least for the majority of PCs where I did this.

I tried this with several Win7 PCs in a row (all of which I do not have access to any more). It might be that the same error also occured when NOT loading the image package before my first try with "pkg install". But I am not 100% sure about this.

You are right, I am also not 100% sure if this behavior was present with older Octave releases. I just have never encountered it before -> maybe no regression (but also maybe it is a regression)

If there are wishes what should be tested (on a Win7 machine) to further nail down this problem, then please let me know a detailed description of the necessary steps. (Because it takes quite a long time to fully un-install Octave, re-install Octave, install the image package...)

 Hartmut <hardy>
Sat 01 Dec 2018 11:45:47 AM UTC, comment #1: 

Was the 2.8.0 image package loaded before trying the first time?

I recognize your description of events for loaded packages with binary modules; the 'permission denied" error is thrown because binary modules in memory (+ probably symbol table) prevent removal of the pertinent .oct files.

Before "pkg install" of a package with binary models, the pkg command should first unload the package and then issue a "clear -f" (or maybe just clear the binary modules from the package in question, but AFAICS "clear -f" doesn't harm other binary modules).

I don't know if this is already implemented in pkg.m but doesn't work, or if it should be implemented. In case of the latter it doesn't seem to difficult to fix IMO.

Status => need info (just to be sure)
Release > dev
Item group - IMO should be "Unexpected error", not "Regression" but let's wait for Hartmut's confirmation

Philip Nienhuis <philipnienhuis>
Group Member
Sat 01 Dec 2018 10:17:32 AM UTC, original submission:  

This happened on two different PCs with Win7:

  • I did a fresh install of Octave 4.4.1 release (64bit Win installer from website), this comes with image-2.8.0 package included.
  • I put the newest image package release zip-file into Octave's working directory (image version 2.8.1 from Forge website)
  • The (update) installation of the image package now throws an error message on the first attempt. But just repeating the command (without any changes at all) the second run then succeeds:



>> pkg install image-2.8.1.tar.gz
couldn't delete directory C:\Octave\OCTAVE~2.1\lib\octave\packages\image-2.8.0\x86_64-w64-mingw32-api
-v52: Permission denied
error: called from
    uninstall at line 124 column 11
    install at line 199 column 9
    pkg at line 437 column 9
>> pkg install image-2.8.1.tar.gz
warning: directory C:\Octave\OCTAVE~2.1\share\octave\packages\image-2.8.0 previously lost
For information about changes from previous versions of the image package, run 'news image'.
>>


Is this a problem with the pkg command? I think this worked fine with older versions of Octave (also under Win7).

 Hartmut <hardy>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by philipnienhuis
  • -email is unavailable- added by philipnienhuis
  • -email is unavailable- added by hardy (Submitted the item)

    •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 6 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2018-12-01 philipnienhuis StatusNeed Info Confirmed
        Carbon-Copy- Added jwe
    2018-12-01 philipnienhuis Carbon-Copy- Added philipnienhuis
    2018-12-01 philipnienhuis Item GroupRegression Unexpected Error or Warning
    2018-12-01 philipnienhuis StatusNone Need Info
        Release4.4.1 dev

    Back to the top

    Copyright © 2024 Free Software Foundation, Inc.
    Verbatim copying and distribution of this entire article is permitted in any medium, provided this notice is preserved.
    The Levitating, Meditating, Flute-playing Gnu logo is a GNU GPL'ed image provided by the Nevrax Design Team.
    Page source code

    Powered by Savane 3.14-f13d.
    Corresponding source code