bugGNU Octave - Bugs: bug #49712, Cannot unload oct file from a SWIG...

 
 

bug #49712: Cannot unload oct file from a SWIG generated wrapper

Submitter:  None
Submitted:  Sat 26 Nov 2016 10:20:07 PM UTC
   
 
Category:  Interpreter Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Incorrect Result
Status:  Works For Me Assigned to:  None
Originator Name:  Originator Email:  -email is unavailable-
Open/Closed:  * Closed Release:  * 4.0.3
Operating System:  * GNU/Linux Fixed Release:  None
Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Sun 27 Nov 2016 06:18:37 PM UTC, comment #6: 

Hi Mike,

Thank you so much for your answer.
It's indeed coming from the wrapper side. I will look into it for sure in more details.

Thanks a lot for the very responsive and efficient follow up! :)

Anonymous
Sun 27 Nov 2016 05:32:15 PM UTC, comment #5: 

The reason for what you are seeing with swig-generated wrappers is that the "module" function "swig_test" calls mlock, which forces it to stay loaded in memory.

The following works for me with your example:


>> swig_test
>> test
0
>> test
1
>> test
2
>> clear -f
>> test
3
>> munlock swig_test
>> clear -f
>> test
0


I can't say why the swig authors decided to mlock this function, whether it will break anything if this is done, or if removing the call to mlock in the wrapper code will break anything.

I think we can say that everything is working as expected here, what you are seeing is swig's complex usage of an oct file containing multiple functions, using autoload to push them into the symbol table, and mlock to keep itself loaded.

If you can find ways to improve the swig interface to Octave, certainly go for it, but there are zero or very few Octave developers involved in developing or working with swig at all.

Mike Miller <mtmiller>
Group Member
Sun 27 Nov 2016 03:05:46 PM UTC, comment #4: 

Hi John,

Thanks for your answer, but my issue is slightly different. Indeed, your explanations match my understanding of the documentation, which mentions the 'autoload' syntax and the symlink approach.
Both work for me, and I have no trouble reproducing the example you posted (either by using 'autoload' as you did, or by creating a symlink 'myotherfun.oct' pointing to 'myfun.oct')

The issue raises when I use swig. When I load a oct module based on a swig wrapper, then I cannot unload it.

As described in my post below, the only way I found so far to call some code from the library while still by able to unload it is to use the symlink approach (I did not try the 'autoload', but I expect it to work).
By doing so, I can call from Octave a function defined in the oct library (based on a swig wrapper) and in this specific case Octave unloads the library properly.
On the contrary, if I directly load the compiled oct, I cannot unload the oct library.

However, if I want to create a wrapped C++ class from Octave or if the wrapped function returns a class, I could not get it working without loading directly the oct file.
When using a symlink, if the function returns a wrapped class, Octave crashes when I try to call it.
If I want to create a wrapped class from Octave, I even don't know how to proceed with symlinks.

So, I wonder if there is a way to unload a swig based oct library. I have trouble extending the autoload/symlink approach while keeping the wrapper fully functional. And I understand that a oct with multiple functions defined may cause some unloading issues.

Anonymous
Sun 27 Nov 2016 01:09:13 PM UTC, comment #3: 

If your .oct file defines multiple functions, then Octave will only unload the .oct file if all the functions defined in the .oct file are cleared.  You should be able to do that by clearing them individually or by using "clear functions".

At least that works for me with a .oct file generated from


#include <iostream>
#include <octave/oct.h>

DEFUN_DLD (myfun, , , "")
{
  static double foo = 0.0;

  foo += 1.0;

  return ovl (foo);
}

DEFUN_DLD (myotherfun, , , "")
{
  return ovl ("foobar!");
}



octave:1> mkoctfile myfun.cc
octave:2> autoload ("myotherfun", "/home/jwe/myfun.oct")
octave:3> myotherfun
ans = foobar!
octave:4> myfun
ans =  1
octave:5> myfun
ans =  2
octave:6> clear functions
octave:7> myfun
ans =  1
octave:8> myotherfun
ans = foobar!
octave:9> clear myfun
octave:10> myfun
ans =  2
octave:11> clear myfun myotherfun
octave:12> myfun
ans =  1


John W. Eaton <jwe>
Group administrator
Sun 27 Nov 2016 12:14:09 PM UTC, comment #2: 

Hi Mike,

Thanks a lot for the fast feedback. Interesting results; most likely is seems bound to my setup.

I attached a simple test case:
- swig_test.hpp / swig_test.cpp are the files to be wrapped
- swig_test.i is the swig interface file
- wrapper.cpp is the wrapper generated by swig (3.0.10) through the command: swig -c++ -octave -o wrapper.cpp swig_test.i
- To generate the oct file I ran:
mkoctfile -std=c++11 --no-gnu-unique -o swig_test.oct swig_test.cpp wrapper.cpp

Then, in Octave, it followed these steps:

> swig_test

2 swig_ref objects ('cvar' and 'swig_test') appeared in the workspace.

> test
> test
> test

Printed 0, then 1, then 2.

> clear all

Then the workspace was cleared.

> test

And here I got a 3 in the output while I expected the command not to be found. It shows that the library was still loaded.

This is fully repeatable on my setup.

The only way I could get it working is if I create a symlink 'test.oct' pointing to 'swig_test.oct'. Then calling 'test' from  Octave and performing a 'clear all' works as expected. But I don't see how I can extend this if C++ classes are involved.


Am I missing something obvious? If you need more details, please feel free to ask.

(file #39080, file #39081, file #39082, file #39083)

Anonymous
Sun 27 Nov 2016 07:20:51 AM UTC, comment #1: 

This seems to work for me.

I have tried both a native oct file and a swig-generated oct file, and both of them call the destructor of a static object when I run either "clear all" or "clear foo". Compiling with or without -fno-gnu-unique makes no difference (that option is news to me).

Can you give a complete example that shows what problem you are seeing? Because it seems to be unloading the oct file correctly for me.

Mike Miller <mtmiller>
Group Member
Sat 26 Nov 2016 10:20:07 PM UTC, original submission:  

Hello,

I experimented wrapping a few pieces of C++ code with SWIG (3.0.10).
It works quite nicely: I could compile without issue the code using mkoctfile and ran it easily from Octave.

When I generate a 'test.oct' file, I can load it by running "test" in the Octave interpreter. Then some objects are visible in the workspace and I can play around.

However, when I run a clear all or clear test the *Octave
workspace is cleared but the oct library remains loaded*. It means for instance that I need to restart Octave each time I rebuild the oct file.

The issue is straightforward to notice if the underlying C++ code handle some static variables.
By adding a few debug prints into the Octave sources, I could confirm that 'dlclose' is not called.

I'm running Ubuntu 16.04.4, with g++ 5.4.
I set the --no-gnu-unique flag when invoking mkoctfile to avoid unique symbols and getting the oct library flagged as NODELETE.

To reproduce, wrapping an empty 'void f()' with SWIG in a 'test' module is enough.
An attempt to reload the oct library in Octave after a clear all and a recompilation of the oct library will lead to the following message:
warning: library test.oct not reloaded due to existing references

As it goes beyond the description covered in the Octave documentation, I'm not sure if I'm missing something or if it's an Octave limitation.

Many thanks




Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #39080:  swig_test.cpp added by None (120B - text/x-c++src)
file #39081:  swig_test.i added by None (80B - application/octet-stream)
file #39082:  swig_test.hpp added by None (84B - text/x-c++hdr)
file #39083:  wrapper.cpp added by None (89KiB - text/x-c++src)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by jwe (Posted a comment)
  • -email is unavailable- added by mtmiller (Posted a comment)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 8 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2016-11-27 mtmiller Open/ClosedOpen Closed
    2016-11-27 None Attached File- Added swig_test.cpp, #39080
        Attached File- Added swig_test.i, #39081
        Attached File- Added swig_test.hpp, #39082
        Attached File- Added wrapper.cpp, #39083
    2016-11-27 mtmiller CategoryNone Interpreter
        Item GroupNone Incorrect Result
        StatusNone Works For Me

    Back to the top

    Powered by Savane 3.13-caa5.
    Corresponding source code