bugGNU Octave - Bugs: bug #55667, Conflict between package namespace...

 
 

bug #55667: Conflict between package namespace and function name

Submitter:  Guillaume <gyom>
Submitted:  Thu 07 Feb 2019 01:37:46 PM UTC
   
 
Category:  Interpreter Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Matlab Compatibility
Status:  None Assigned to:  None
Originator Name:  Guillaume Open/Closed:  * Open
Release:  * dev Operating System:  * Any
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Fri 18 Dec 2020 02:08:34 PM UTC, comment #17: 

I also came across this bug, while trying to port a larger Matlab application to Octave.

Overall, it was a surprisingly smooth experience. However, the namespace clashes mean we have to rewrite quite some part of the application, which is a pretty big obstacle.

Ulf Lorenz <ulflorenz>
Fri 15 Feb 2019 07:17:37 PM UTC, comment #16: 

Done: bug #55717.

Mike Miller <mtmiller>
Group Member
Fri 15 Feb 2019 07:06:47 PM UTC, comment #15: 

I have some questions, so yes, it would probably be best to open a new bug report and I'll comment there.

John W. Eaton <jwe>
Group administrator
Fri 15 Feb 2019 06:22:53 PM UTC, comment #14: 

Should we fork that to a separate bug report?

Mike Miller <mtmiller>
Group Member
Fri 15 Feb 2019 06:22:19 PM UTC, comment #13: 

Yeah, if you are happy to implement a special feature like that to handle it, I will gladly give up my workaround and use that instead. I'd be happy to use either an m-file hook in the +pkg directory or some C++ registration call that inserted such a hook into the namespace resolution.

I just need either the complete string passed in, or the equivalent "subs" struct that would be passed in to subsref/subsasgn, created from the parts that come after the "pkg.", whichever makes the most sense.

Mike Miller <mtmiller>
Group Member
Fri 15 Feb 2019 05:56:55 PM UTC, comment #12: 

Mike: In comment #6, you wrote:

If there were a way to somehow register a proper classdef +py namespace with a dynamic name resolution hook, I could do that.

Maybe that's what we should do instead of relying on additional indexing of the results of function calls for this purpose.  Then we could decide to deprecate that feature (or not) and it wouldn't affect your code.

I think the implementation would be fairly straightforward.  If a package (+pkg Matlab-style package) had this feature enabled, any name prefixed with "pkg." that is not explicitly part of the package results in a call to a special dispatch function.  I think the code that needs to change to provide this feature is the bit in cdef_package::cdef_package_rep::meta_subsref that currently errors if the "nm" part of pkg.nm is undefined.

So, how should we go about registering this special catch-all function?

The simplest thing seems to me to just recognize a special file inside a +pkg directory.  That way you could just define a _CATCH_ALL_.m (or .oct or .mex) file function there that would do the job.   I'm not sure I like the name _CATCH_ALL_, but something like that wouldn't even require any special treatment in the parser/lexer.

John W. Eaton <jwe>
Group administrator
Thu 14 Feb 2019 11:49:59 PM UTC, comment #11: 

Another place where there may be a compatibility issue is for calling static class methods.  As I understand it, if a class named "foo" has a static member function named "bar", then


Foo.bar


should call the static member function.  What does Matlab do for


Foo.bar.baz


?  Is this an error?  If not, what happens with the ".baz" part of this expression?

Octave currently ignores the ".baz" part, which I think is wrong.    If we did the same as for other functions, then we would call foo.bar and then index the result with ".baz".  It may be possible, though I think it will be more complicated than making it an error.

John W. Eaton <jwe>
Group administrator
Thu 14 Feb 2019 11:17:17 PM UTC, comment #10: 

Mike: I don't want to remove this functionality.  Right now, I'm just trying to understand whether Octave's current behavior causes compatibility issues.

Am I remembering correctly that Matlab does not allow a function foo that returns a struct to be indexed as


foo.bar


?

In any case, what does Matlab do if you have a function foo and ask


which foo.bar


And what does it do if you have a function bar.m in a package directory named +foo and you ask


which foo.bar.baz


?  I'm guessing both fail?

John W. Eaton <jwe>
Group administrator
Tue 12 Feb 2019 09:51:02 PM UTC, comment #9: 

I still think it's important that we do not get rid of the useful 'foo.index' with no replacement for that loss of functionality.

Mike Miller <mtmiller>
Group Member
Tue 12 Feb 2019 09:02:35 PM UTC, comment #8: 

@JWE:
If foo().index and foo(some,arguments).index could be kept working, even if foo.index can't (for a function foo), I think that would help much.
Regarding your example with dir(), I don't see the reason for assuming nargout == 0 in a call like foo().index ...

Well, I counted less than 100 lines in package optim containing

function_call(arguments)(index)

or

value(index)(index) .

But using subsref() or some streamlined version of it as an alternative would not improve readability...

Olaf Till <i7tiol>
Group Member
Thu 07 Feb 2019 09:07:37 PM UTC, comment #7: 

John, thanks for the explanation. Yes i hpe that Octave, through repeated indexing, would be more efficient with RAM usage and faster (even if just a little) and if I understand you correctly that is indeed the case.

In Matlab it's often a cumbersome affair to slowly peel off complicated data structures layer by layer in order to arrive at some deeply buried variable, compared to Octave where it can often be done in just one command. Same goes for function output, e.g., structs or struct arrays.

Philip Nienhuis <philipnienhuis>
Group Member
Thu 07 Feb 2019 06:16:39 PM UTC, comment #6: 

I would appreciate some help in understanding the tradeoffs and options here. I do understand the conflict being discussed here and why it's a problem with overloaded names. What I would like help with is understanding how I should proceed with the Python interface.

I need a way for any call that begins with "py." to be dynamically interpreted as a lookup on a Python function or class name. If there were a way to somehow register a proper classdef +py namespace with a dynamic name resolution hook, I could do that.

At the moment, I have an Octave class named "@py". So calling "py.int" instantiates an Octave @py object and then does a subsref on it, at which point I can pass that on to my Python name resolution function.

If we take away inline dot indexing of function calls, I am not sure how else I can implement this in Octave right now. Any suggestions? We can certainly take this offline and discuss in more detail if needed.

Mike Miller <mtmiller>
Group Member
Thu 07 Feb 2019 05:57:23 PM UTC, comment #5: 

Philip, if you are asking is something like


y = x(2:4)(2);


automatically transformed to x(3), then the answer is no.  It is just performed as a series of indexing calls that generate internal temporary variables.  So ultimately it's not much different from writing


tmp = x(2:4);
y = tmp(2);


except that in the first case, the internal temporary is not entered into the symbol table and may be cleared as soon as it is no longer needed.

But Octave also attempts to manage data array slices without making copies, so indexing like this should not create additional copies of the original array.

John W. Eaton <jwe>
Group administrator
Thu 07 Feb 2019 05:37:25 PM UTC, comment #4: 


From the point of view of the implementation, I'd be glad to eliminate the possibility of further indexing function calls


Probably it'll bring some pain for recoding, sure. But another question: does further or repeated indexing save memory?
In Matlab, I'd have to slice and sub-slice numeric/struct/cell arrays using intermediate variables for each indexing step and that sometimes takes considerable RAM (esp. cell arrays); I figured that one of the advantages of repeated indexing would be avoiding these intermediate storgae steps. Am I right there?

Philip Nienhuis <philipnienhuis>
Group Member
Thu 07 Feb 2019 04:28:58 PM UTC, comment #3: 

Another issue is that "foo" and "foo()" may both be function calls.  Would it help if, in order to further index a function call, you had to write foo().index instead of just foo.index?  At least then you would not have to look up "foo" as a function when evaluating "foo.index".  Or would this not eliminate all possible conflicts?

From the point of view of the implementation, I'd be glad to eliminate the possibility of further indexing function calls because it would greatly simplify the indexing code in the Octave interpreter.

This change would cause no trouble for people using Octave to run code developed for Matlab.  But it would probably cause a lot of pain for many Octave users who have come to rely on this feature.

Oh, I just tried


dir.name    ## works, dir is called with nargout == 1
dir().name  ## fails, because when called this way,
            ## dir is called with nargout == 0 and in
            ## that case it doesn't return a struct.


?!?

Is this behavior really what we want?

John W. Eaton <jwe>
Group administrator
Thu 07 Feb 2019 04:06:18 PM UTC, comment #2: 

That's a worry... This is a scenario I had in mind for some code refactoring where I have an octave.m function and want to create +octave/+algebra, +octave/+plot, etc, directories while still keeping octave.m for backward compatibility.

It might look like a contrived example but it still means that it prevents you to name a function with any namepace that might be in your path (matlab and containers by default so far with Octave only).

While I like the "function calls may be further indexed" extension, I was wondering whether this particular scenario could be detected and a workaround be added.

Guillaume <gyom>
Thu 07 Feb 2019 02:59:14 PM UTC, comment #1: 

Have we finally found the example that breaks our "function calls may be further indexed" extension to the language?

John W. Eaton <jwe>
Group administrator
Thu 07 Feb 2019 01:37:46 PM UTC, original submission:  

There is a conflict between package namespaces and functions with the same name, due to the "chaining" feature Octave implements to reference the output of a function.

Use the following toy example:


>> mkdir +octave
>> system ("echo function octave, disp('package'); > +octave/octave.m");
>> system ("echo function octave, disp('function'); > octave.m");


In Octave:


>> octave
function
>> octave.octave
function
error: can't perform indexing operations for <unknown type> type


and Matlab:


>> octave
function
>> octave.octave
package
>> import octave.*
>> octave
package



Guillaume <gyom>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by ulflorenz (Posted a comment)
  • -email is unavailable- added by i7tiol (Posted a comment)
  • -email is unavailable- added by jwe (Posted a comment)
  • -email is unavailable- added by gyom (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    No changes have been made to this item

    Back to the top

    Powered by Savane 3.13-f8d8.
    Corresponding source code