bugGNU Octave - Bugs: bug #53856, MEX file in a private directory of...

 
 

bug #53856: MEX file in a private directory of a class

Submitted by:  Guillaume <gyom>
Submitted on:  Mon 07 May 2018 10:38:14 AM UTC  
 
Category:  Interpreter Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Matlab Compatibility
Status:  Fixed Assigned to:  None
Originator Name:  Guillaume Open/Closed:  Closed
Release:  dev Operating System:  GNU/Linux

Add a New Comment (Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

( Jump to the original submission)

Tue 15 May 2018 04:58:27 PM UTC, comment #17: 

I think we can close this as fixed now. The additional problem of a mex or oct function appearing after the directory has already been  loaded is covered by bug #40824.

Mike Miller <mtmiller>
Project Administrator
Tue 15 May 2018 09:20:15 AM UTC, comment #16: 

Than you, I can confirm it now works and solve the issue I reported here. Thank you also for pushing the fix to stable.

Guillaume <gyom>
Tue 15 May 2018 05:18:22 AM UTC, comment #15: 

OK, I think I see what is happening.  It's just luck that one type of file (.m, .oct, or .mex) is preferred when more than one type exists in a private directory because we weren't properly noting the multiple types.  Does the following patch fix this part of the problem for you?

  http://hg.savannah.gnu.org/hgweb/octave/rev/cc40e47d3a44

It still won't handle the case of a .mex or .oct file appearing after the .m file has been used because the out_of_date_check code doesn't properly handle that.  But it should at least make it so that the function we call doesn't depend on the order of the list of files returned by the directory reading function.

John W. Eaton <jwe>
Project Administrator
Mon 14 May 2018 09:52:05 PM UTC, comment #14: 

I still get the following behavior (with Rik's mex_private example posted here, freshly extracted):

$ cd mex_private/private/
$ mkoctfile-4.4.0 --mex myprivfcn.c
$ cd  ..
$ octave -q
>> version -hgid
ans = c8f49ee7a687
>> tst_privfcn
error: Calling the m-file!
error: called from
    myprivfcn at line 2 column 3
    tst_privfcn at line 4 column 3
>> unlink private/myprivfcn.m;
>> tst_privfcn
error: myprivfcn: Calling the MEX-file!
error: called from
    tst_privfcn at line 4 column 3

And after restoring the m-file function, I looked at the '__dump_load_path__' function you mentioned:

>> __dump_load_path__


*** private functions in ./private:

  myprivfcn (m)



Mike Miller <mtmiller>
Project Administrator
Mon 14 May 2018 09:08:34 PM UTC, comment #13: 

I stlil can't explain why the .m file would be called if the mex file already exists.  But for the case of compiling the mex file after the m file has already been executed once, I think there are two problems.

First, we weren't looking at the timestamp of the private directory, so the load path was not updated if the private subdirectory changed.  This changeset should fix that problem:

  http://hg.savannah.gnu.org/hgweb/octave/rev/c8f49ee7a687

I am able to see a difference with the _dump_load_path_ function.  Previously, after compiling the .mex file, the private function map still had the .m file.  Now it will be updated to show the .mex file instead.

Second, the function out_of_date_check in symtab.cc doesn't check to see whether a .oct or .mex file has appeared when checking whether a .m file is out of date.  I don't have a fix for that yet because I don't know whether that check is always appropriate in the out_of_date_check function or if it should be done separately (or possibly that the entire logic here should be revamped).

However, with the first changeset above, the following now works for me when it previously failed:

addpath ("/path/to/mex_private")
tst_privfcn  ## calls .m file
cd /path/to/mex_private/private
mex myprivfcn.c
cd /some/where
tst_privfcn  ## still calls .m file
clear functions
tst_privfcn  ## now calls .mex file

Previously, the clear functions command had no effect because the load path was never updated.

John W. Eaton <jwe>
Project Administrator
Fri 11 May 2018 03:11:49 PM UTC, comment #12: 

Yes, those are the places to look.

John W. Eaton <jwe>
Project Administrator
Fri 11 May 2018 03:08:31 PM UTC, comment #11: 

There are a number of bug reports that seem related in one way or another: bug #40824, bug #45444, bug #46281, bug #47693, bug #48925

Are libinterp/corefcn/fcn-info.cc and libinterp/corefcn/load-path.cc the place to look at for this?

Guillaume <gyom>
Wed 09 May 2018 03:23:27 PM UTC, comment #10: 

Thanks for your tests - it's a funny one so I'm glad some of you reproduced it. As I mentioned in my initial comment, I've got some more complicated examples involving mex files and private directory where it works fine so it's a strange interaction with something else I cannot figure out yet.

I don't have a simple example at hand but I' have also seen situations where private m-files functions where not detected by Octave (without any mex file involved) so there is also probably another dimension to this error.

One way to reproduce the error is through Mike's docker image:
  https://hub.docker.com/r/mtmiller/octave-snapshot/

Guillaume <gyom>
Wed 09 May 2018 01:52:15 AM UTC, comment #9: 

I'll try to figure out what is going on with this problem.

John W. Eaton <jwe>
Project Administrator
Tue 08 May 2018 10:27:51 PM UTC, comment #8: 

I just updated and installed revision bc5f225bc578 and I still see the same thing on my system, the m-file is always called even if the mex file exists when Octave starts.

>> tst_privfcn
error: Calling the m-file!
error: called from
    myprivfcn at line 2 column 3
    tst_privfcn at line 4 column 3
>> cd private
>> myprivfcn
error: myprivfcn: Calling the MEX-file!
>> cd ..
>> tst_privfcn
error: Calling the m-file!
error: called from
    myprivfcn at line 2 column 3
    tst_privfcn at line 4 column 3

Mike Miller <mtmiller>
Project Administrator
Tue 08 May 2018 09:16:30 PM UTC, comment #7: 

Rik's test code works for me as well if I compile the MEX file before executing the test function but it fails if I start Octave without the MEX file compiled, then do

John W. Eaton <jwe>
Project Administrator
Tue 08 May 2018 09:14:50 PM UTC, comment #6: 

I compiled the mex file in the shell with Octave not running. I basically followed the steps you posted in comment #3.

Here is an exact transcript of what I am doing:

$ tar -xf ~/Downloads/mex_private.tar.gz
$ cd mex_private/private/
$ mkoctfile-5.0.0 --mex myprivfcn.c
$ cd ..
$ octave-5.0.0 --norc --no-history --silent
octave:1> tst_privfcn
error: Calling the m-file!
error: called from
    myprivfcn at line 2 column 3
    tst_privfcn at line 4 column 3

Mike Miller <mtmiller>
Project Administrator
Tue 08 May 2018 09:00:51 PM UTC, comment #5: 

Boy, that's weird.  I have re-confirmed that I get the print out from the MEX file with 5.0.0 which is installed, and also using run-octave in a development directory.  The HG ID for this is

parent: 25360:bc5f225bc578 tip
eliminate some global accesses to the breakpoint table

from today.

If I start with no .mex file in the private directory, then I get the output from the m-file.  With Octave still running, if I then use a shell and mkoctfile to compile the .mex file, I still get the output from them-file.  No amount of 'clear -f' or 'clear -a' will make Octave see the new .mex file.  However, if I quit Octave and re-start then the printout is from the .mex file.

So, it may be something about scanning for .mex files only once at startup.  I also tried 'rmpath mex_private; addpath mex_private' hoping to force a re-scan, but it didn't work.

If the .mex file is present, however, when you start Octave then you are seeing a different issue.

Rik <rik5>
Project Administrator
Tue 08 May 2018 08:29:11 PM UTC, comment #4: 

On my system Octave prints the message from the m-file, both with the original example and with Rik's example in comment #3.

Mike Miller <mtmiller>
Project Administrator
Tue 08 May 2018 05:09:17 PM UTC, comment #3: 

I tested the situation in comment #2, with a mex file in a private directory which is not associated with a classdef file, and it works for me.

I've attached the entire directory structore and files that I used in mex_private.tar.gz.

To test

1) tar xf mex_private.tar.gz
2) cd mex_private/private
3) mkoctfile --mex myprivfcn.c
4) cd ..
5) octave -f
6) tst_privfcn

On my system it prints out a message from the MEX file, not the message from the m-file.

(file #44124)

Rik <rik5>
Project Administrator
Tue 08 May 2018 09:22:20 AM UTC, comment #2: 

And same thing with a classdef file. The issue is actually not class related and occurs with the following:

.
├── callprivate.m
└── private
    ├── mysubfunction.c
    ├── mysubfunction.cc
    ├── mysubfunction.m
    ├── mysubfunction.mex
    └── mysubfunction.oct

where mysubfunction.* is as before while callprivate.m is:

function callprivate ()
  mysubfunction ();
endfunction

callprivate () calls the M function and not its compiled alternative.

Guillaume <gyom>
Tue 08 May 2018 08:54:02 AM UTC, comment #1: 

I tried with an oct file instead of a mex file and the same thing happens.

Guillaume <gyom>
Mon 07 May 2018 10:38:14 AM UTC, original submission:  

I observed errors by which M functions in a private directory are not overshadowed by an associated MEX file. I have got plenty of examples where it does work but it seems to fail with the toy example below.

Just compile the MEX file with mkoctfile --mex mysubfunction.c and run:

octave:1> a = myclass ();
octave:2> mymethod (a)
error: Calling the M-file!
error: called from
    mysubfunction at line 2 column 3
    mymethod at line 2 column 3

@myclass
├── myclass.m
├── mymethod.m
└── private
    ├── mysubfunction.c
    ├── mysubfunction.m
    └── mysubfunction.mex

function m = myclass
  m = class (struct ('a',1), 'myclass');
endfunction

function mymethod(a)
  mysubfunction ();
endfunction

function mysubfunction ()
  error ("Calling the M-file!");
endfunction

#include "mex.h"

void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[]) {
  mexErrMsgTxt("Calling the MEX-file!");
}

Guillaume <gyom>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #44124:  mex_private.tar.gz added by rik5 (3KiB - application/gzip)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by jwe (Posted a comment)
  • -email is unavailable- added by mtmiller (Posted a comment)
  • -email is unavailable- added by rik5 (Updated the item)
  • -email is unavailable- added by gyom (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can add your encouragement to it.
    This task has 0 encouragements so far.

    Only project members can vote.

     

     

     

    Follow 7 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2018-05-15 mtmiller StatusIn Progress => Fixed
        Open/ClosedOpen => Closed
    2018-05-14 jwe StatusConfirmed => In Progress
    2018-05-10 rik5 StatusWorks For Me => Confirmed
    2018-05-08 rik5 Attached File- => Added mex_private.tar.gz, #44124
        Item GroupUnexpected Error => Matlab Compatibility
        StatusNone => Works For Me

    Back to the top


    Powered by Savane 3.5