bugGNU Octave - Bugs: bug #45864, mkoctfile is unable to overwrite...

 
 

bug #45864: mkoctfile is unable to overwrite oct-files that have been loaded on Windows

Submitter:  Mario Barbati <mariob>
Submitted:  Wed 02 Sep 2015 03:56:50 PM UTC
   
 
Category:  Octave Function Severity:  1 - Wish
Priority:  3 - Low Item Group:  Feature Request
Status:  In Progress Assigned to:  None
Originator Name:  Mario Barbati Open/Closed:  * Open
Release:  * dev Operating System:  * Microsoft Windows
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Wed 19 Dec 2018 11:03:29 AM UTC, comment #16: 

I forgot about this bug, but bug #55143 is clearly a duplicate of this one.
I'll try to pick up here again in the Xmas holidays.

In the mean time I think this issue deserves a higher priority than low plus severity of just "Wish"; IMO it isn't a feature request but rather a bug.

Philip Nienhuis <philipnienhuis>
Group Member
Tue 18 Dec 2018 10:15:54 PM UTC, comment #15: 

I could do some testing on Win7 for this bug, but I'm not deeply into the "C side" of Octave. Could you tell me what precise steps I should do (on Win) to help the progress of this bug? (I could easily use the image package, and the two previously mentioned .cc functions bwlabel and bwlabeln for this purpose.)

Hartmut <hardy>
Mon 04 Jul 2016 12:03:03 AM UTC, comment #14: 

It's been almost too long to remember what was going on in this bug. I think I was trying to come up with a routine that could clear all of the functions in a compiled .oct file by name that we could drop into mkoctfile. This would allow it to try to unload before overwriting the .oct file.

So I think what needs testing is

1. start with a .cc file that defines more than one DEFUN_DLD
2. compile it with mkoctfile
3. declare the autoload for the second function if necessary
4. check that both functions work
5. try to compile it again, mkoctfile should fail to overwrite
6. call 'clearoct("NAME.oct")' with the literal name of the file
7. try to compile NAME.cc again, does it work now?

Mike Miller <mtmiller>
Group Member
Thu 30 Jun 2016 12:04:27 PM UTC, comment #13: 

Retagging "Patch submitted" to "In progress" because the patch doesn't seem to be a complete solution.

Lachlan Andrew <lachlan>
Tue 22 Sep 2015 05:41:50 PM UTC, comment #12: 

@Mike: is this along the lines you suggest?

>> pth
pth = C:\Programs\Octave\Octave-4.1.0+_w64\lib\octave\packages\image-2.4.1\x86_64-w64-mingw32-api-v50+

>> autoload ("bwlabel", [pth filesep "bwlabeln.oct"])

>> which bwlabel
'bwlabel' is a function from the file C:\Programs\Octave\Octave-4.1.0+_w64\lib\octave\packages\image-2.4.1\x86_64-w64-mingw32-api-v50+\bwlabeln.oct

>> which bwlabeln
'bwlabeln' is a function from the file C:\Programs\Octave\Octave-4.1.0+_w64\lib\o
ctave\packages\image-2.4.1\x86_64-w64-mingw32-api-v50+\bwlabeln.oct

>> which clearoct
'clearoct' is a function from the file C:\Users\philip\MyDocs\octave\clearoct.m

>> clearoct ("bwlabeln")

>> which bwlabeln
'bwlabeln' is a function from the file C:\Programs\Octave\Octave-4.1.0+_w64\lib\octave\packages\image-2.4.1\x86_64-w64-mingw32-api-v50+\bwlabeln.oct

>> which bwlabel
'bwlabel' is a function from the file C:\Programs\Octave\Octave-4.1.0+_w64\lib\octave\packages\image-2.4.1\x86_64-w64-mingw32-api-v50+\bwlabeln.oct
>>


Similar goes for _COM_.oct (using com_get, com_set).

I suspect I have to do something with the autoload func before this works but I never used that so I'm, well, clueless here.

Philip Nienhuis <philipnienhuis>
Group Member
Tue 15 Sep 2015 09:47:15 PM UTC, comment #11: 

Still on my list...
Last week I had a quick try with _COM_.oct in the windows package and there it didn't seem to work.
After next weekend I'll give it another try with a.o., bwlabeln.oct and maybe some geometry package .octfiles.

Philip Nienhuis <philipnienhuis>
Group Member
Fri 11 Sep 2015 11:28:16 AM UTC, comment #10: 

Placing it in mkoctfile took a little more thought than I had at the time. The problem is mkoctfile.m passes its arguments off blindly to mkoctfile the program, some logic would need to be added to figure out which argument is the file name and turn it into the corresponding oct-file. Not trivial, and maybe not even a good idea.

For now I was hoping someone could just test by calling this function manually to see if it does indeed free up the file lock on Windows allowing you to recompile.

The image package has bwlabeln.oct, which contains the functions bwlabel and bwlabeln, as one example.

Mike Miller <mtmiller>
Group Member
Fri 04 Sep 2015 07:10:03 AM UTC, comment #9: 

Sure. Just 2 Q:

- Any good examples of such an .oct-file? e.g., in some OF package?

- Where to put the function? (= how to "link" it to mkoctfile so that it works automatically?)

Philip Nienhuis <philipnienhuis>
Group Member
Fri 04 Sep 2015 12:00:18 AM UTC, comment #8: 

The attached function works for me, given an oct-file name it clears all associated DEFUNs from Octave, looks to me like the file is completely unloaded. Philip can you test on Windows with an oct-file that contains more than one DEFUN?

(file #34812)

Mike Miller <mtmiller>
Group Member
Thu 03 Sep 2015 11:55:27 PM UTC, comment #7: 

I was about to test also, I see jwe already pushed that one: http://hg.savannah.gnu.org/hgweb/octave/rev/c36f06ef7539

So remaining is to clear functions associated with an oct-file in mkoctfile before attempting to compile it? I took a look at the return value of the autoload function, could be used to search for symbols in a given oct filename that has more than one DEFUN.

Mike Miller <mtmiller>
Group Member
Thu 03 Sep 2015 07:31:08 PM UTC, comment #6: 

@JWE:
I tried your patch on Windows and it works fine. It does allow removing autoloaded functions from memory so that the shared .dlls they live in can be overwritten by mkoctfile.

BTW it never occurred to me that the persistent file lock on binary modules could be due to an Octave bug (or rather, I figured it was a Windows bug). Thanks for enlightening me

Philip Nienhuis <philipnienhuis>
Group Member
Wed 02 Sep 2015 09:21:55 PM UTC, comment #5: 

OK, as I said on IRC, I think it is autoload functions that are not being cleared properly when using "clear foo".  So it's not a counting issue.  It's that you can only properly clear functions that have the same name as the .oct file.  The fix seems simple, it's just to call "clear_autoload_function (force)" in the function symbol_table::fcn_info::fcn_info_rep::clear_user_function.

I propose the attached patch to fix that problem.

(file #34797)

John W. Eaton <jwe>
Group administrator
Wed 02 Sep 2015 08:05:53 PM UTC, comment #4: 

Hmm, when I do the same for _voronoi_ (the only function defined in _voronoi_.oct) I also see 5 instances of _voronoi_.oct in the output of lsof.  But clearing _voronoi_ does result in _voronoi_.oct being unloaded.  So I think the 5 instances in the output of lsof is not the problem.

John W. Eaton <jwe>
Group administrator
Wed 02 Sep 2015 08:00:53 PM UTC, comment #3: 

Weird.  If I look at qr.oct, for example, it defines 5 functions.  Calling qr appears to load the file 5 times (as shown by "lsof | grep qr.oct").  Then if I do


clear qr qrupdate qrinsert qrdelete qrshift


lsof still shows 5 instances of qr.oct.

That looks like a bug to me.  I think the intent was to unload the shared library if all the functions it defines are cleared.

John W. Eaton <jwe>
Group administrator
Wed 02 Sep 2015 07:46:31 PM UTC, comment #2: 

Mario did ask about this on #octave first, and jwe suggested that he provide a patch or report this in some way, so here we are! :)

I think his suggestion is reasonable, that mkoctfile should attempt to unload the oct-file before compiling it to allow it to be overwritten on operating systems where the file is locked when in use.

I can create a simple oct-file on Linux, prove to myself that the file is open by Octave after the first time the function is called, and is no longer open if I run "clear -f funcname". If this is done automatically by mkoctfile, this might help the majority of cases.

It might not help in cases where a single oct-file defines more than one function, I can't seem to get the oct-file to unload on Linux unless I do a complete "clear -f".

Mike Miller <mtmiller>
Group Member
Wed 02 Sep 2015 07:13:29 PM UTC, comment #1: 

This happens if you've used the .oct file in an Octave command; then the operating system will lock the file so that it cannot be overridden.

Remedy:

=>  Close & restart Octave (that'll unlock the file), then run mkoctfile again.

Please report back if this solves your problem.

BTW before reporting these kind of issues as a bug, better first ask on the mailing list (help-octave@octave.org).

Philip Nienhuis <philipnienhuis>
Group Member
Wed 02 Sep 2015 03:56:50 PM UTC, original submission:  

once created a octfile

mkoctfile foo.cc

is not possible compile it again without closing octave, the error is:


mkoctfile foo.cc

Permission denied


the procedure is run clear:

clear foo

then is possible compile again.


Will be usefull insert this command in mkoctfile function

Mario Barbati <mariob>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #34812:  clearoct.m added by mtmiller (292B - text/x-objcsrc)
file #34797:  diffs.txt added by jwe (397B - text/plain)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by apjanke
  • -email is unavailable- added by hardy (Posted a comment)
  • -email is unavailable- added by lachlan (Posted a comment)
  • -email is unavailable- added by jwe (Posted a comment)
  • -email is unavailable- added by mariob (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 13 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2019-10-18 apjanke Carbon-Copy- Added -email is unavailable-
    2018-12-20 mtmiller Carbon-CopyRemoved 80942 -
    2016-06-30 lachlan StatusPatch Submitted In Progress
    2015-09-11 mtmiller Summaryproblem with mkoctfile mkoctfile is unable to overwrite oct-files that have been loaded on Windows
    2015-09-04 mtmiller Attached File- Added clearoct.m, #34812
    2015-09-02 jwe StatusNeed Info Patch Submitted
    2015-09-02 jwe Attached File- Added diffs.txt, #34797
    2015-09-02 mtmiller CategoryNone Octave Function
        Severity3 - Normal 1 - Wish
        Priority5 - Normal 3 - Low
        Item GroupNone Feature Request
        Release4.0.0 dev
    2015-09-02 philipnienhuis StatusNone Need Info

    Back to the top

    Powered by Savane 3.13-4448.
    Corresponding source code