bugGNU Octave - Bugs: bug #29476, Parser incompatibility for...

 
 

bug #29476: Parser incompatibility for keywords against Matlab implementation, infinite loop

Submitter:  Paul Ortyl <ortylp>
Submitted:  Thu 08 Apr 2010 04:48:38 PM UTC
   
 
Category:  Interpreter Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Incorrect Result
Status:  Wont Fix Assigned to:  None
Originator Name:  Open/Closed:  * Closed
Release:  * 3.3.51 Operating System:  * GNU/Linux
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Mon 16 Aug 2010 07:53:43 AM UTC, comment #16: 

I'm closing this bug since no new fruits have come off it, and there seems to be a conclusion that it's best to do nothing here. The hangup no longer exists.

Jaroslav Hajek <highegg>
Thu 29 Apr 2010 05:53:25 PM UTC, comment #15: 

I don't object to having a special mode that limits keywords to just those in Matlab (i.e., enabled when using --traditional).  However, that mode setting must not affect files that are clearly written for Octave (files distributed with Octave, or that use Octave-style comments, etc.).  The question is how to determine when to apply the different parsing rules.

Is it really sufficient to just use a path-based mechanism?

I am not interested in trying to allow handles to keywords at all, so you would only be able to create handles to Octave-specific keywords in "Matlab mode".  When or how could that cause trouble?


John W. Eaton <jwe>
Group administrator
Fri 09 Apr 2010 11:24:24 PM UTC, comment #14: 

Ok. I misunderstood slightly and I didn't realize that structure elements already worked. So, I agree functions/methods and global variables seem to be the problem.

Global variables could be handled by a function as you suggest. Another option is to use the new namespace syntax e.g. global.do since "global" is already a keyword in both octave and matlab. [I'm not saying that global.do works in Matlab, I don't have access to a Matlab new enough to have namespace support]

feval() works, but it would be more elegant if handles worked.

_do = @do
_do()

It does seem to me like it would be useful to mix and match traditional and octave code. Selectivity of features is too complex. The choice should be "strict Matlab" or not. I would go after a path-based mechanism rather than try to guess based on content.

What about the new Matlab keywords that we don't have ("parfor", "spmd")?

You previously asked whether Matlab allows functions with keyword names and I don't think that was answered. It does not:

+verbose+
a.m:
function this = a()
 this.do = @do;
 function this = do(this, in)
 this = this+in;
 end
end

b.m:
function this = b()
 this.for = @for;
 function this = for(this, in)
 this = this+in;
 end
end
-verbose-

Then in Matlab R2007a:

+verbose+

>> a                      

ans =
    do: @a/do

>> b

??? Error: File: /tmp/test7/b.m Line: 3 Column: 14
Illegal use of reserved keyword "for".
-verbose-

Judd Storrs <judd>
Fri 09 Apr 2010 10:13:03 PM UTC, comment #13: 

I don't really want to invent new syntax for this feature.  I would just like to think about whether there is a way that we can make the compatibility freaks happy without doing too much damage (i.e., I don't want to eliminate the extended keywords completely).

There is no problem with parsing structure elements now.  You should already be able to write x.foo = 42, for example.

The only case I can think of where you might need to access a symbol that has the same name as one of the extra Octave keywords is if you need to call a function or a global variable that is defined with the keyword name.  For functions, we already have feval.  I guess we would need some way to get a (global) variable value via a function.  That should be simple enough to do.

Or, maybe it is not needed at all, since if you started Octave with --traditional to allow this feature to work in .m files at all, then maybe you should not be able to use the extended set of Octave keywords at the command line either?  And if you need to write a function that must access global variables or functions that have keyword names, then you can just write it using the --traditional mode.

John W. Eaton <jwe>
Group administrator
Fri 09 Apr 2010 09:16:25 PM UTC, comment #12: 

(If you're reading this on the mailing list it looks like the backslashes were eaten by the savannah mailer from my last comment)


\do = 4
\end = \do
t.\endwhile = 4


Judd Storrs <judd>
Fri 09 Apr 2010 09:13:25 PM UTC, comment #11: 

One last thought. Outside traditional mode maybe \<keyword> could do what is wanted? i.e.

\do = 4
\end = 5

Judd Storrs <judd>
Fri 09 Apr 2010 09:04:45 PM UTC, comment #10: 

It's possible even if some functions can be marked as traditional mode and keywords were allowed for variable names, the a struct return value with an illegal field is still possible. e.g. t.do or t.endfunction.

So, there would probably still need to be some way to access t.<keyword> outside of traditional mode. Maybe quoting? i.e. t.do in traditional mode and t."do" in awesome mode?

Judd Storrs <judd>
Fri 09 Apr 2010 08:51:59 PM UTC, comment #9: 

Is the goal to support Matlab files without modification or to make it possible to optionally use "do" as a variable? This also came up in my attempt to use niftimatlib and I ended up just doing a find replace to turn all "do" into "_do".

Perhaps one other way to deal with this would be if traditional mode could be triggered based on presence of some sort of canary file in a directory. So that if I have a bunch of Matlab files in say, "mymatlabfiles" that I intend to be compatible with both platforms, I could create "mymatlabfiles/.octave_traditional_mode" or something similar to help the interpreter keep octave extensions out of that code by accident? You'd still have to mark the directory but it could be easier than marking each file or function?

Judd Storrs <judd>
Fri 09 Apr 2010 08:16:34 PM UTC, comment #8: 

I suppose we could also modify the is_keyword_token function to not treat Octave's extra keywords as special in certain cases.  If we do this, then I think even with --traditional we should accept the additional keywords if we are parsing an Octave function.  The difficulty is in deciding when we are parsing an Octave function.  I would guess at least all files that are in Octave's "system" path, but which others?  Octave Forge package files (all or just some)?  Files that, when we look at them (not taking into account the additional keywords, of course) could obviously not be parsed by Matlab.  I guess that would include any file that had comments starting with #, for example.  But would that only count if the # token appeared before seeing any of the additional keywords?  Or should there be some other way of marking a file (say in a comment) as an "Octave" or "Matlab" file?  Would it be sufficient to have one conditional ("strict Matlab syntax rules" vs. "normal Octave syntax rules") or should it be possible to enable individual features selectively?  Or is this all just too complicated? 

If is_keyword_token is changed, some corresponding change would also need to be made to the is_keyword function in src/lex.ll.

John W. Eaton <jwe>
Group administrator
Fri 09 Apr 2010 05:46:52 AM UTC, comment #7: 

You should be able to do this by editing octave.gperf, and altering or removing the keywords you need. You then need to regenerate oct-gperf.h; make will do it, but you must have gperf installed. Bison and flex may also be needed for subsequent regenaration of lex.cc and oct-parse.cc.

This description also makes it clear that this would be currently difficult or impossible to do at runtime, hence turning out the keywords at --traditional is out of question for technical reasons, even if it was a good idea.

Note that functions may stop working due to this; currently there seem to be none in core Octave, but there may be in the future.

Jaroslav Hajek <highegg>
Thu 08 Apr 2010 08:37:58 PM UTC, comment #6: 

The compatibility with octave-forge packages in --traditional mode was also my concern...  Coincidentally packages used by me do not use 'do' keyword.

This is not place to discuss 'goal of the Octave'.  There are only two things I would like to express now:

1. I have chosen Octave over 10 ten years ago BECAUSE it is/was Matlab compatible and I am grateful, that I could use Octave as a student, use now at home and trying to use at work, where Matlab is the target platform.  In all these three use environments Matlab compatibility had very high priority in order to not divide but combine development efforts and make it independent of target platform.

2. Where in the code can I disable do...while for my local build.

Thanks

Paul Ortyl <ortylp>
Thu 08 Apr 2010 08:16:54 PM UTC, comment #5: 

I don't think that disabling do-until loops is a good solution, even (or maybe especially) if it is only done for --traditional. Although there does not appear to be any code in Octave itself that uses do-until, there are a number of uses in Octave Forge packages.  Disabling do-until loops would mean that all of that code would not work in --traditional mode.

You seem to have the mistaken impression that a goal of the Octave project is for it to be an exact clone of Matlab.  Although compatibility does have a relatively high priority, I don't think that 100% compatibility is the target.

John W. Eaton <jwe>
Group administrator
Thu 08 Apr 2010 07:44:02 PM UTC, comment #4: 

OK, I was wrong, 'do' is not a keyword in Matlab, see
http://www.mathworks.com/access/helpdesk/help/techdoc/ref/iskeyword.html

In the case 'do' is not a keyword in Matlab, then in --traditional mode it should not be in octave either.

Paul Ortyl <ortylp>
Thu 08 Apr 2010 07:05:35 PM UTC, comment #3: 

Are you sure that Matlab allows you to create a function with the same name as a keyword?  Can you create a function called "for" or "if"?

This is a separate issue from the fact that Octave and Matlab have slightly different sets of keywords.  In Octave, "do" and "until" are keywords, but I don't think they are in Matlab.

John W. Eaton <jwe>
Group administrator
Thu 08 Apr 2010 06:47:29 PM UTC, comment #2: 

In short: Matlab (tested with R2008a) accepts the code and runs it correctly.

In the example, which is boiled down version of legacy module I have to use an cannot modify, the 'do' is in context of definition of function 'prototype'.  In this context, forced by keyword 'function', these two letters 'do' cannot be keyword 'do' from the do...until loop.  It means that a function named 'do' can be created overriding the keyword 'do'.  It has the consequence, that no do...until loop may be used in this file.  As long as the language 'specification' does not prohibit overriding of keywords the code is correct.  Taken, that the reference implementation is Matlab(R) then the masking of keyword by function name is allowed.

I must admit, that the concept of defining function with name theoretically 'reserved' to keyword is scary idea... but Matlab does not prohibit that.

I would expect this 'functionality' to work at least in --traditional mode.

Paul Ortyl <ortylp>
Thu 08 Apr 2010 05:40:43 PM UTC, comment #1: 

In Octave, "do" is a keyword (for the do until loop syntax) so you can't create a function or function handle using that name.

Although I am able to reproduce the hanging behavior with Octave 3.3.51, a copy of Octave built from the development sources returns immediately with a syntax error.

"The problem is usage of keyword 'do' as function name. As long as it is unambiguous, and in the example above IT IS, the keyword was accepted as function name and allowed in the reference to be assigned."

I don't understand what you mean by "the keyword was accepted as function name".  In what instance does Octave allow a function to have the same name as a keyword?

John W. Eaton <jwe>
Group administrator
Thu 08 Apr 2010 04:48:38 PM UTC, original submission:  

Here below is a stripped down piece of code, which parses and works in Matlab (R2008a), fails in Octave 3.2.4, fails and hangs Octave 3.3.51.

------------------------------
function this = a()

 this.do = @do;

 function this = do(this, in)
  this = this+in;
 end
end
-----------------------------

=============================
error: function handles may not refer to keywords
parse error near line 3 of file a.m

  syntax error

>>>   this.do = @do;

                ^

error: function handles may not refer to keywords
parse error near line 3 of file a.m

  syntax error

>>>   this.do = @do;

                ^
================================

The problem is usage of keyword 'do' as function name.  As long as it is unambiguous, and in the example above IT IS, the keyword was accepted as function name and allowed in the reference to be assigned.

Octave 3.2.4 reports the error and returns to prompt, version 3.3.51 does not return but hangs taking 100% of CPU time.

Paul Ortyl <ortylp>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by judd (Posted a comment)
  • -email is unavailable- added by highegg (Posted a comment)
  • -email is unavailable- added by jwe (Posted a comment)
  • -email is unavailable- added by ortylp (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 2 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2010-08-16 highegg StatusNone Wont Fix
        Open/ClosedOpen Closed

    Back to the top

    Powered by Savane 3.13-02a9.
    Corresponding source code