bugGNU Octave - Bugs: bug #47763, load: function handle to user...

 
 

bug #47763: load: function handle to user function uses full path, should just search load path

Submitter:  Guillaume <gyom>
Submitted:  Fri 22 Apr 2016 05:11:51 PM UTC
   
 
Category:  Octave Function Severity:  3 - Normal
Priority:  5 - Normal Item Group:  None
Status:  Need Info Assigned to:  None
Originator Name:  Guillaume Open/Closed:  * Open
Release:  * dev Operating System:  * GNU/Linux
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Thu 07 Sep 2017 02:12:46 AM UTC, comment #9: 

I have created a patch for this problem. It is in file patch_bug47763.diff.

As I mentioned before some of the problem was connected to lazy function handle evaluation. So I implemented it as well in this patch. Basically when no function is found a octave_user_function is created with a new bool property dummy_function. Calling octave_user_function::call() on this function causes a lazy feval evaluation using the function name. I also added an octave_value::is_dummy_function() query to tell if a certain function is a dummy function.

I also added a new test folder with tests for the lazy function handle evaluation. I did not add tests for this exact problem,  however I manually tested the original sequence as well as a case where there is ambiguity (like in comment Comment #6) and everything worked for me.

(file #41750)

Piotr Held <jsoh425>
Fri 01 Sep 2017 12:18:36 AM UTC, comment #8: 

This behavior doesn't seem to be limited to loading function handles. Creating the function handle 'f' with simply


f = @foo;


yields same results as loading it from file. If created without 'foo' in load path the resulting handle has no file associated with it and is always lazily evaluated. If 'foo' is in path during creation then 'foo' is eagerly assigned to 'f' and can be called even if 'foo' is not in the load-path anymore.

Piotr Held <jsoh425>
Mon 25 Apr 2016 04:58:16 PM UTC, comment #7: 

Just to add that, both Matlab and Octave behave in the same way when a function handle is created from a function in the search path and later on removed from said search path:


>> functions(f)
ans =
    function: 'myfoo'
        type: 'simple'
        file: '/tmp/foo2/myfoo.m'
>> f()
foo2
>> rmpath /tmp/foo2
>> f()
foo2
>> myfoo
Undefined function or variable 'myfoo'.


Guillaume <gyom>
Mon 25 Apr 2016 11:06:17 AM UTC, comment #6: 

Thanks again, Mike. Here is an example where the full path is used when there is an "ambiguity"; myfoo() is available twice in the search path (either displaying 'foo1' or 'foo2'):


>> addpath /tmp/foo1
>> myfoo()
foo1
>> f=@myfoo;
>> addpath /tmp/foo2
>> myfoo()
foo2
>> f()
foo1
>> save foo.mat f
>> clear f; load foo.mat
>> f()
foo1
>> rmpath /tmp/foo1
>> clear f; load foo.mat
>> f()
foo2


So the behavior when loading a function handle from a mat-file seems to be:

  • if there is a function corresponding to the full path and that path is in the search path, create a function handle to it
  • otherwise, ignore the full path and create a function handle to the first function with that name in the search path if it exists
  • if it doesn't exist, create a function handle without warning to a nonexisting function with that name (a call to foo() returns "Undefined function or variable 'myfoo'.") and use the first function in the search path when one becomes available:



>> rmpath /tmp/foo1
>> rmpath /tmp/foo2
>> clear f; load foo.mat
>> functions(f)
ans =
    function: 'myfoo'
        type: 'simple'
        file: ''
>> f()
Undefined function or variable 'myfoo'.
>> addpath /tmp/foo1
>> f()
foo1
>> addpath /tmp/foo2
>> f()
foo2


The last part is less important and related to this "won't fix" bug #31821

Guillaume <gyom>
Fri 22 Apr 2016 06:18:11 PM UTC, comment #5: 

Ok, so you're saying that Matlab does also save the full path to the m-file, but it doesn't use it at all when the file is loaded? Did you do a "save -ascii" in Matlab to confirm this? Is there any way to test that Matlab actually doesn't use the full path for anything when loading? The cases you've shown suggest that it doesn't.

One more case I've thought of is what if the handle is saved as a handle to a builtin function, and loaded in a Matlab session where that function is shadowed by a user function on the path?

Mike Miller <mtmiller>
Group Member
Fri 22 Apr 2016 06:14:47 PM UTC, comment #4: 

With a core function (bar()):


>> type /tmp/bar.m

function bar
disp('Hello')
>> addpath /tmp
>> % create bar.mat with Matlab
>> cd ~
f=@bar;
>> functions(f)

ans =

    function: 'bar'
        type: 'simple'
        file: '/tmp/bar.m'

>> save bar.mat f -v6
>> % load when file is where expected and in the path
>> addpath /tmp
>> load bar.mat
>> functions(f)

ans =

    function: 'bar'
        type: 'simple'
        file: '/tmp/bar.m'

>> f()
Hello
>> % load when file is where expected but not in the path
>> rmpath /tmp
>> load bar.mat
>> functions(f)

ans =

    function: 'bar'
        type: 'simple'
        file: '/usr/local/MATLAB/R2016a/toolbox/matlab/specgraph/bar.m'

>> f()
Error using bar (line 47)
Not enough input arguments.

>> % load when file is elsewhere in the path
>> movefile('/tmp/bar.m',pwd)
>> load bar.mat
>> functions(f)

ans =

    function: 'bar'
        type: 'simple'
        file: '/home/foo/bar.m'

>> f()
Hello


With a core function, the behaviour is the same apart from many warning to be displayed:


Warning: Function plot has the same name as a MATLAB builtin. We suggest you rename the function to avoid a potential name conflict.


Re your change of summary: as mentioned in my previous comment, the issue is with load and not save (there's a separate bug report for that, bug #43215).

Guillaume <gyom>
Fri 22 Apr 2016 05:56:57 PM UTC, comment #3: 

Thanks, Mike. For the first part of your comment, the full path to the function file is saved in the MAT file (by Matlab). When loading a function handle variable, this full path should be ignored and the search path should be used instead.

I'll do the other tests shortly.

Guillaume <gyom>
Fri 22 Apr 2016 05:36:36 PM UTC, comment #2: 

Confirmed here. The octave_fcn_handle::load_... and octave_fcn_handle::save_... functions store the full path to the function file, and restore it when loaded. It sounds like Matlab only stores the short name to the function file and does a lookup on the current load path, whether it's the same actual function file or not doesn't matter.

Can you also test on Matlab with a user function that shadows a core function file and one that shadows a builtin? I don't have access to Matlab so I don't know for sure what are builtins, but I would guess "plot" is a function file, and probably "quit" is a builtin. The same tests would be useful, load when the m-file is in the same directory, when the directory is removed from the load path, and when the file is moved to a different directory in the load path.

Mike Miller <mtmiller>
Group Member
Fri 22 Apr 2016 05:14:06 PM UTC, comment #1: 

And, unsurprisingly, foo.m is:


function foo
  disp('Hello');
end


Guillaume <gyom>
Fri 22 Apr 2016 05:11:51 PM UTC, original submission:  

When loading a MAT-file containing a function handle, it doesn't work if the path to the function has changed (but is still in the function search path). See the following piece of code:


>> % have foo.m in /tmp
>> type /tmp/foo.m
>> addpath /tmp
>> % create foo.mat with Matlab
>> cd ~
>> f=@foo;
>> functions(f)
>> save foo.mat f -v6
>>
>> % load when file is where expected and in the path
>> addpath /tmp
>> load foo.mat
>> functions(f)
>> f()
>>
>> % load when file is where expected but not in the path
>> rmpath /tmp
>> load foo.mat
>> functions(f)
>> f()
>>
>> % load when file is elsewhere in the path
>> movefile('/tmp/foo.m',pwd)
>> load foo.mat
>> functions(f)
>> f()


Octave does:


>> % load when file is where expected and in the path
>> addpath /tmp
>> load foo.mat
>> functions(f)
ans =

  scalar structure containing the fields:

    function = foo
    type = simple
    file = /tmp/foo.m

>> f()
Hello
>> % load when file is where expected but not in the path
>> rmpath /tmp
>> load foo.mat
>> functions(f)
ans =

  scalar structure containing the fields:

    function = foo
    type = simple
    file = /tmp/foo.m

>> f()
Hello
>> % load when file is elsewhere in the path
>> movefile('/tmp/foo.m',pwd)
ans = 1
>> load foo.mat
error: no such file, '/tmp/foo.m'
>> functions(f)
ans =

  scalar structure containing the fields:

    function = foo
    type = simple
    file = /tmp/foo.m

>> f()
error: foo: no longer valid function handle


while Matlab does:


>> % load when file is where expected and in the path
>> addpath /tmp
>> load foo.mat
>> functions(f)

ans =

    function: 'foo'
        type: 'simple'
        file: '/tmp/foo.m'

>> f()
Hello
>> % load when file is where expected but not in the path
>> rmpath /tmp
>> load foo.mat
>> functions(f)

ans =

    function: 'foo'
        type: 'simple'
        file: ''

>> f()
Undefined function or variable 'foo'.

>> % load when file is elsewhere in the path
>> movefile('/tmp/foo.m',pwd)
>> load foo.mat
>> functions(f)

ans =

    function: 'foo'
        type: 'simple'
        file: '/home/foo/foo.m'

>> f()
Hello


So Octave finds the function when it's not in the search path but in its initial location (Matlab doesn't, appropriately) and doesn't find the function if it's in the search path but not in its initial location.

I attach the foo.mat file created with Matlab to this bug report.

Guillaume <gyom>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #41750:  patch_bug47763.diff added by jsoh425 (14KiB - text/x-patch)
file #36980:  foo.mat added by gyom (832B - application/octet-stream)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by jsoh425 (Posted a comment)
  • -email is unavailable- added by mtmiller (Posted a comment)
  • -email is unavailable- added by gyom (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 5 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2017-09-07 jsoh425 Attached File- Added patch_bug47763.diff, #41750
    2016-04-22 mtmiller Summarysaving a function handle to user function saves full path load: function handle to user function uses full path, should just search load path
    2016-04-22 mtmiller StatusNone Need Info
        SummaryLoading a function handle variable from a MAT-file saving a function handle to user function saves full path
    2016-04-22 gyom Attached File- Added foo.mat, #36980

    Back to the top

    Powered by Savane 3.13-f8d8.
    Corresponding source code