bugGNU Octave - Bugs: bug #53856, MEX file in a private directory of...

 
 

bug #53856: MEX file in a private directory of a class

Submitted by:  Guillaume <gyom>
Submitted on:  Mon 07 May 2018 10:38:14 AM UTC  
 
Category:  Interpreter Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Matlab Compatibility
Status:  Fixed Assigned to:  None
Originator Name:  Guillaume Open/Closed:  Closed
Release:  dev Operating System:  GNU/Linux

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

( Jump to the original submission)

Tue 15 May 2018 04:58:27 PM UTC, comment #17:

I think we can close this as fixed now. The additional problem of a mex or oct function appearing after the directory has already been loaded is covered by bug #40824.

Mike Miller <mtmiller>
Project Administrator
Tue 15 May 2018 09:20:15 AM UTC, comment #16:

Than you, I can confirm it now works and solve the issue I reported here. Thank you also for pushing the fix to stable.

Guillaume <gyom>
Tue 15 May 2018 05:18:22 AM UTC, comment #15:

OK, I think I see what is happening. It's just luck that one type of file (.m, .oct, or .mex) is preferred when more than one type exists in a private directory because we weren't properly noting the multiple types. Does the following patch fix this part of the problem for you?

http://hg.savannah.gnu.org/hgweb/octave/rev/cc40e47d3a44

It still won't handle the case of a .mex or .oct file appearing after the .m file has been used because the out_of_date_check code doesn't properly handle that. But it should at least make it so that the function we call doesn't depend on the order of the list of files returned by the directory reading function.

John W. Eaton <jwe>
Project Administrator
Mon 14 May 2018 09:52:05 PM UTC, comment #14:

I still get the following behavior (with Rik's mex_private example posted here, freshly extracted):

And after restoring the m-file function, I looked at the '__dump_load_path__' function you mentioned:

Mike Miller <mtmiller>
Project Administrator
Mon 14 May 2018 09:08:34 PM UTC, comment #13:

I stlil can't explain why the .m file would be called if the mex file already exists. But for the case of compiling the mex file after the m file has already been executed once, I think there are two problems.

First, we weren't looking at the timestamp of the private directory, so the load path was not updated if the private subdirectory changed. This changeset should fix that problem:

http://hg.savannah.gnu.org/hgweb/octave/rev/c8f49ee7a687

I am able to see a difference with the _dump_load_path_ function. Previously, after compiling the .mex file, the private function map still had the .m file. Now it will be updated to show the .mex file instead.

Second, the function out_of_date_check in symtab.cc doesn't check to see whether a .oct or .mex file has appeared when checking whether a .m file is out of date. I don't have a fix for that yet because I don't know whether that check is always appropriate in the out_of_date_check function or if it should be done separately (or possibly that the entire logic here should be revamped).

However, with the first changeset above, the following now works for me when it previously failed:

Previously, the clear functions command had no effect because the load path was never updated.

John W. Eaton <jwe>
Project Administrator
Fri 11 May 2018 03:11:49 PM UTC, comment #12:

Yes, those are the places to look.

John W. Eaton <jwe>
Project Administrator
Fri 11 May 2018 03:08:31 PM UTC, comment #11:

There are a number of bug reports that seem related in one way or another: bug #40824, bug #45444, bug #46281, bug #47693, bug #48925

Are libinterp/corefcn/fcn-info.cc and libinterp/corefcn/load-path.cc the place to look at for this?

Guillaume <gyom>
Wed 09 May 2018 03:23:27 PM UTC, comment #10:

Thanks for your tests - it's a funny one so I'm glad some of you reproduced it. As I mentioned in my initial comment, I've got some more complicated examples involving mex files and private directory where it works fine so it's a strange interaction with something else I cannot figure out yet.

I don't have a simple example at hand but I' have also seen situations where private m-files functions where not detected by Octave (without any mex file involved) so there is also probably another dimension to this error.

One way to reproduce the error is through Mike's docker image:
https://hub.docker.com/r/mtmiller/octave-snapshot/

Guillaume <gyom>
Wed 09 May 2018 01:52:15 AM UTC, comment #9:

I'll try to figure out what is going on with this problem.

John W. Eaton <jwe>
Project Administrator
Tue 08 May 2018 10:27:51 PM UTC, comment #8:

I just updated and installed revision bc5f225bc578 and I still see the same thing on my system, the m-file is always called even if the mex file exists when Octave starts.

Mike Miller <mtmiller>
Project Administrator
Tue 08 May 2018 09:16:30 PM UTC, comment #7:

Rik's test code works for me as well if I compile the MEX file before executing the test function but it fails if I start Octave without the MEX file compiled, then do

John W. Eaton <jwe>
Project Administrator
Tue 08 May 2018 09:14:50 PM UTC, comment #6:

I compiled the mex file in the shell with Octave not running. I basically followed the steps you posted in comment #3.

Here is an exact transcript of what I am doing:

Mike Miller <mtmiller>
Project Administrator
Tue 08 May 2018 09:00:51 PM UTC, comment #5:

Boy, that's weird. I have re-confirmed that I get the print out from the MEX file with 5.0.0 which is installed, and also using run-octave in a development directory. The HG ID for this is

from today.

If I start with no .mex file in the private directory, then I get the output from the m-file. With Octave still running, if I then use a shell and mkoctfile to compile the .mex file, I still get the output from them-file. No amount of 'clear -f' or 'clear -a' will make Octave see the new .mex file. However, if I quit Octave and re-start then the printout is from the .mex file.

So, it may be something about scanning for .mex files only once at startup. I also tried 'rmpath mex_private; addpath mex_private' hoping to force a re-scan, but it didn't work.

If the .mex file is present, however, when you start Octave then you are seeing a different issue.

Rik <rik5>
Project Administrator
Tue 08 May 2018 08:29:11 PM UTC, comment #4:

On my system Octave prints the message from the m-file, both with the original example and with Rik's example in comment #3.

Mike Miller <mtmiller>
Project Administrator
Tue 08 May 2018 05:09:17 PM UTC, comment #3:

I tested the situation in comment #2, with a mex file in a private directory which is not associated with a classdef file, and it works for me.

I've attached the entire directory structore and files that I used in mex_private.tar.gz.

To test

On my system it prints out a message from the MEX file, not the message from the m-file.

(file #44124)

Rik <rik5>
Project Administrator
Tue 08 May 2018 09:22:20 AM UTC, comment #2:

And same thing with a classdef file. The issue is actually not class related and occurs with the following:

where mysubfunction.* is as before while callprivate.m is:

callprivate () calls the M function and not its compiled alternative.

Guillaume <gyom>
Tue 08 May 2018 08:54:02 AM UTC, comment #1:

I tried with an oct file instead of a mex file and the same thing happens.

Guillaume <gyom>
Mon 07 May 2018 10:38:14 AM UTC, original submission:

I observed errors by which M functions in a private directory are not overshadowed by an associated MEX file. I have got plenty of examples where it does work but it seems to fail with the toy example below.

Just compile the MEX file with mkoctfile --mex mysubfunction.c and run:

Guillaume <gyom>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #44124:  mex_private.tar.gz added by rik5 (3KiB - application/gzip)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by jwe (Posted a comment)
  • -email is unavailable- added by mtmiller (Posted a comment)
  • -email is unavailable- added by rik5 (Updated the item)
  • -email is unavailable- added by gyom (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can add your encouragement to it.
    This task has 0 encouragements so far.

    Only project members can vote.

     

     

     

    Follow 7 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2018-05-15 mtmiller StatusIn Progress => Fixed
        Open/ClosedOpen => Closed
    2018-05-14 jwe StatusConfirmed => In Progress
    2018-05-10 rik5 StatusWorks For Me => Confirmed
    2018-05-08 rik5 Attached File- => Added mex_private.tar.gz, #44124
        Item GroupUnexpected Error => Matlab Compatibility
        StatusNone => Works For Me

    Back to the top


    Powered by Savane 3.3