bugGNU Octave - Bugs: bug #60882, error parsing command syntax

 
 

bug #60882: error parsing command syntax

Submitter:  Guillaume <gyom>
Submitted:  Mon 05 Jul 2021 09:33:15 AM UTC
   
 
Category:  Interpreter Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Regression
Status:  Fixed Assigned to:  None
Originator Name:  Guillaume Open/Closed:  * Closed
Release:  * dev Operating System:  * Any
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Tue 17 Oct 2023 11:15:52 AM UTC, comment #50: 

The issue has disappeared in the code analyzer report on GitHub.

Markus Mützel <mmuetzel>
Group administrator
Fri 13 Oct 2023 03:24:18 PM UTC, comment #49: 

Thanks for checking.

Markus Mützel <mmuetzel>
Group administrator
Fri 13 Oct 2023 01:15:01 PM UTC, comment #48: 

Markus: your change seems fine to me.  Thanks.

John W. Eaton <jwe>
Group administrator
Fri 13 Oct 2023 01:01:28 PM UTC, comment #47: 

In hindsight, I should probably opened a new report for this.

I pushed the following change which I hope does the "right thing"™:
https://hg.savannah.gnu.org/hgweb/octave/rev/9afc383bb60a

Somebody who better understands the parser should probably double-check though.

Markus Mützel <mmuetzel>
Group administrator
Fri 13 Oct 2023 12:34:39 PM UTC, comment #46: 

Code scanning on GitHub pointed to a possible use after free issue with one of the changes from here:
In libinterp/parse-tree/oct-parse.yy lines 3473ff:

            if (parfor)
              {
                delete lhs;
                delete expr;
                delete maxproc;
                delete body;

                bison_error ("invalid syntax for parfor statement");
              }

            m_lexer.mark_as_variables (lhs->variable_names ());


If `parfor` is true, `lhs` is deleted (`bison_error` is not noreturn). In that case, lhs would still be dereferenced by `lhs->variable_names ()`.

Should `m_lexer.mark_as_variables (lhs->variable_names ());` be in the else branch of `if (parfor)`? Or should that line be executed before `lhs` is deleted?

Markus Mützel <mmuetzel>
Group administrator
Thu 16 Dec 2021 08:51:21 PM UTC, comment #45: 

Thanks for catching that oversight.  I fixed it in this changeset http://hg.savannah.gnu.org/hgweb/octave/rev/9e0421bd8bad.

Rik <rik5>
Group administrator
Thu 16 Dec 2021 08:02:12 PM UTC, comment #44: 

var.m is using the '.-' syntax at line 219 and a deprecation warning is displayed in dev.

Anonymous
Tue 07 Dec 2021 03:23:11 PM UTC, comment #43: 

I pushed a documentation change and NEWS entry about the no longer allowed whitespaces between increment/decrement operator and the variable they affect here:
https://hg.savannah.gnu.org/hgweb/octave/rev/7ecd07721cca

Markus Mützel <mmuetzel>
Group administrator
Tue 23 Nov 2021 03:29:18 PM UTC, comment #42: 

After this change, the "**" and ".**" operators are marked as  deprecated.

http://hg.savannah.gnu.org/hgweb/octave/rev/2ba4758654ca

Closing as fixed, since I think the original problem has been fixed.

If we want to continue the discussion about what to do for the compound OP= assignment operators, we can do that in the discourse discussion here:  https://octave.discourse.group/t/parsing-command-style-function-call-syntax/1414.  I copied the text from comment #39 there since it should help understanding the current situation with command syntax parsing vs. operators in Octave that are not available in Matlab.

John W. Eaton <jwe>
Group administrator
Mon 22 Nov 2021 01:32:35 AM UTC, comment #41: 

Is there anything else to do here?  The last thing mentioned was that we would start deprecating '**' and '.**' exponent operators.  That isn't specific to this bug report so this one could probably be closed.

Rik <rik5>
Group administrator
Wed 25 Aug 2021 12:51:34 AM UTC, comment #40: 

Per Octave Developer Call today, we will deprecate


**
.**


but keep


.+
.-
+verbatim+

For the latter, can we tag these so they throw an Octave-only warning the first time they are used?

Rik <rik5>
Group administrator
Tue 24 Aug 2021 10:14:10 AM UTC, comment #39: 

Rik: until recent changes, when executing code at the command line, Octave checked whether a symbol was defined as a variable and if so, did not attempt to parse a statement as a command-style function call.  I believe that is also what Matlab used to do.  But now, for consistency in behavior, Matlab uses the same syntax-only rules to parse statements.  It doesn't consult the current workspace at parse time.  The "used as variable and later as function" message happens after parsing is already done, at the time the code is executed.

I think the only question left is whether to handle our extended set of operators (++, +=, etc.) specially or in a fully Matlab-compatible way when parsing command syntax.  Since Matlab doesn't have += (for example), I believe that an expression like


foo +=


is parsed as a command-style function call regardless of what follows the +=.  If we do the same for Octave, then it is impossible to use += as an operator unless the expression is written with no space after the identifier:


foo+=something    ## OP=
foo+= something   ## OP=
foo +=something   ## command-style function call
foo += something  ## command-style function call


That seems bad to me, so I left the special treatment of these operators in Octave so that the only one that causes command-style function parsing is


foo +=something


similar to the way that both Octave and Matlab handle other binary operators (i.e., only the "foo +something" is handled as command-style function syntax).

Since ++ is a unary operator and + by itself is both a binary and prefix unary operator, the rules are a little different.  In Matlab, the parsing of expressions involving repeated + characters is a bit weird:


foo ++            %% command-style function call
foo ++ something  %% command-style function call
foo++something    %% expression: foo + (+something)
foo++ something   %% expression: foo + (+something)
foo ++something   %% command-style function call


See also

https://octave.discourse.group/t/parsing-command-style-function-call-syntax/1414/4

where I posted a test function to verify current behavior of Matlab and Octave.  Note that in the results that Markus posted for Matlab, the "OK" doesn't mean that Octave and Matlab behavior is the same, just that we know what the behaviors are.  Looking at the function itself shows that there are currently differences between  Octave and Matlab for the following operators:


@
\
~
.+
.-
.**
!=
**
++
--

all the OP= operators.


It would probably be fairly painless to phase out and eventually remove


.+
.-
.**
**


We are already eliminating the problem with backslash as a continuation character outside of character strings.  I think we can make the behavior of @ and ~ compatible with Matlab.  Changing != (and !) to be compatible would be very unpleasant.

Since ++ and -- are not binary operators anyway, we can get closer to the Matlab behavior if we reject them as two-character tokens in some cases depending on the surrounding context, return just '+' (or '-') from the lexer, and continue parsing beginning with the next '+' (or '-') character from the input stream.  That should provide compatibility with Matlab by parsing expressions like "foo++something" as "foo + (+something)".

John W. Eaton <jwe>
Group administrator
Sat 21 Aug 2021 09:50:24 PM UTC, comment #38: 

I assume this is related, but I get an error for this code:


octave:2> myvar = 5
myvar = 5
octave:3> myvar .^2
error: myvar used as variable and later as function
octave:4> myvar .^ 2
ans = 25


In Octave 6.3 there is no error for the above code.

Rik <rik5>
Group administrator
Fri 06 Aug 2021 05:33:43 PM UTC, comment #37: 

I should clarify that "x ++ + 3" evaluates to 3 when x is originally 0. I neglected to mention the initial value of x. In general it evaluates to x+3 overall, and increments x as a side-effect. This behavior is actually consistent with C++ and Java, so might not be wrong though it is ill-advised to write such an expression in production code.

Anonymous
Fri 06 Aug 2021 07:39:10 AM UTC, comment #36: 

Matlab and Octave differentiate between command syntax and function syntax (see section 11.13 of the current manual).
`x ++` is interpreted as command syntax. Thus, it requires that `x` is valid function identifier. If it is not, an error is emitted.
`y = x ++` cannot be interpreted in command syntax (because of the assignment which is impossible in command syntax). Thus, it is treated as function syntax. That means `++` cannot be a function argument (or the user would need to write `y = x('++')`). Thus, it is treated as an operator.

`x ++ + 3` evaluating to 3 might be a bug. IIUC, it should be evaluated the same as `x('++', '+', '3')` which should result in an error if `x` is not a valid function identifier.

Markus Mützel <mmuetzel>
Group administrator
Thu 05 Aug 2021 06:26:48 PM UTC, comment #35: 

I request clarification. The statement "x ++" causes an error but the command "y = x ++" does not, which evaluates to "y = x; x += 1;". Nor does the admittedly pathological statement "x ++ + 3" cause an error, which evaluates to 3. That is, "x ++" with the space only causes an error when it occurs by itself, not when embedded in a bigger statement. Is this behavior intended?

Anonymous
Fri 30 Jul 2021 06:14:17 PM UTC, comment #34: 

There was a bunch of work on this and discussion over on the Discourse site.  Is there more to be done or can this report be closed?

Rik <rik5>
Group administrator
Wed 21 Jul 2021 05:51:00 PM UTC, comment #33: 

Imho, we don't need to aim for full syntax compatibility for the operators that Octave has but Matlab does not have.

From a user point of view, it is possible to write code that is compatible with Octave and with Matlab without too much effort at all: Just avoid command syntax if the arguments look like operators.

Markus Mützel <mmuetzel>
Group administrator
Wed 21 Jul 2021 05:40:04 PM UTC, comment #32: 

I wrote:


I think it is better if there is not special treatment at the command line, so I undid that part of my previous change, here:


and forgot the link to the change.  It is here:

http://hg.savannah.gnu.org/hgweb/octave/rev/ef865fcd4611

John W. Eaton <jwe>
Group administrator
Wed 21 Jul 2021 05:38:00 PM UTC, comment #31: 

BTW, this trouble all started when I was working on handling the new "arguments" block syntax and I decided to try to eliminate some duplicated information in the parser and interpreter.  Instead of storing lists of pending variables (function parameters and identifiers found on the LHS of assignment operators) in the lexer and storing identifiers in the symbol table, I thought it would be best to just tag the symbols in the symbol table as (expected) variables based on syntax alone.  A mistake in that change caused the bug reported here.  (The mistake was entering "diary" as a variable when it appeared in an expression like "job.diary = ...", even though "diary" by itself here isn't a variable.)

Now I think I'm just confused about what we should be doing here, because I just made a change so that we are once again treating


a=1;
b=2;
a -b


at the command line differently from what happens in a function or script because my change made Octave look at the symbol table to decide whether "a" is a variable when parsing "a -b", and if it is, to not handle that statement as a command.  But according to comment #15, current versions of Matlab are no longer doing that?  I think it is better if there is not special treatment at the command line, so I undid that part of my previous change, here:

I also think it is best if we can parse the command-style syntax completely based on syntax, so I would prefer to stick with that change.  But it does present some compatibility problems.  If we use syntax alone and opt for full Matlab compatibility, then none  of the OP= operators can work properly with simple identifiers at the beginning of an expression because Matlab will always parse statements like "foo -= bar" as commands (please correct me if that's not the case).

John W. Eaton <jwe>
Group administrator
Wed 21 Jul 2021 05:29:22 PM UTC, comment #30: 

Thanks, that's good news because it is much simpler to not have to recognize expressions like that as commands.

John W. Eaton <jwe>
Group administrator
Wed 21 Jul 2021 05:13:30 PM UTC, comment #29: 

That doesn't seem to be working in Matlab R2021a either:

>> type +a/b

function b(arg1, arg2)
disp(arg1);
disp(arg2);
end
>> a.b('foo', 'bar')
foo
bar
>> a.b foo bar
 a.b foo bar
     ↑
Invalid expression. Check for missing multiplication operator, missing or unbalanced delimiters, or other syntax error. To construct matrices, use brackets instead of parentheses.


Markus Mützel <mmuetzel>
Group administrator
Wed 21 Jul 2021 05:07:19 PM UTC, comment #28: 

Rik: RE comment #27, an expression like


foo() ++


wouldn't be interpreted as a command anyway because with the way the parser is now, that can only happen when the token before the operator is a simple identifier.

This raises another question about how Matlab works though...

Is a name like PKG.FCN allowed as the name of a command?  In other words, is it possible to use the full name of a package function as a command?  If so, then Octave's parser still isn't handling this syntax "correctly".

John W. Eaton <jwe>
Group administrator
Wed 21 Jul 2021 04:45:08 PM UTC, comment #27: 

I fixed the one example in core octave in _strip_html_tags_.m.  See http://hg.savannah.gnu.org/hgweb/octave/rev/ee84485289ce.

I used grep to look for ' --' and ' ++' sequences in the scripts directory and that was the only one I found.

Rik <rik5>
Group administrator
Wed 21 Jul 2021 04:23:12 PM UTC, comment #26: 

Definitely this should go in the NEWS file for Octave 7.

And documentation should be updated to strongly recommend cuddling postfix operators (--, ++) with their variable for clarity.

For testing, I do think it will simply come down to loading a package and running regression tests.  I would hope that programmers have not used a space between variables and post-fix operators extensively.

Rik <rik5>
Group administrator
Wed 21 Jul 2021 04:22:48 PM UTC, comment #25: 

Re comment #24: That change only affects the .m file parser. It shouldn't affect C or C++ code.

Markus Mützel <mmuetzel>
Group administrator
Wed 21 Jul 2021 04:20:43 PM UTC, comment #24: 

Re Comment #22, agreed, it would make sense to require postfix ++ and -- operators to immediately follow their variable without a space.

For uses of "foo ++" within the Octave codebase and in Octave Forge packages, they should probably be changed to "foo++" or "foo += 1" as preferred.

Please note the following potentially problematic uses of postfix notation within the Octave codebase, to be changed if required:


libgui/qterminal/libqterminal/win32/QWinTerminalImpl.cpp:728:        begin.rx () --;
libgui/qterminal/libqterminal/win32/QWinTerminalImpl.cpp:734:      end.rx () ++;

libgui/qterminal/libqterminal/win32/QWinTerminalImpl.cpp:742:      begin.rx () --;
libgui/qterminal/libqterminal/win32/QWinTerminalImpl.cpp:748:      end.rx () ++;

scripts/help/private/__strip_html_tags__.m:63:      groups(end, 2) ++;


Anonymous
Wed 21 Jul 2021 04:15:43 PM UTC, comment #23: 

Thanks.
That might be the best way forward.

With Octave 6, `x --` was interpreted as decrement operation.
IIUC, the same syntax will be interpreted as command syntax in Octave 7. This might break code that currently uses something like that.
I gather there is no easy way to deprecate that syntax for a few versions before we finally change its meaning.

IIUC, syntax like that can be fixed easily by removing the whitespace (`x--`) or by forcing it to an expression (e.g. with `(x --)`). That should have the same meaning in Octave 6 and in Octave 7.

Is there a way we can reach out to users and developers (e.g. of Octave Forge packages) to give them some head start of that upcoming change?

Markus Mützel <mmuetzel>
Group administrator
Wed 21 Jul 2021 03:58:08 PM UTC, comment #22: 

I pushed the following changeset to make Octave treat all binary ops the same whether they are in Matlab or not. 

http://hg.savannah.gnu.org/hgweb/octave/rev/da6e5914ddaf

So now


x-=2    %% operator
x-= 2   %% operator
x -= 2  %% operator
x -=2   %% command syntax


Attempting to handle postfix ++ and -- as operators just by looking at syntax and when they are preceded by spaces is complicated.  If we do the same as shown above, then it is impossible to use common use of '--' to indicate the end of options:


foo -- -this-is-not-an-option  %% should be a command?



John W. Eaton <jwe>
Group administrator
Wed 21 Jul 2021 01:08:41 PM UTC, comment #21: 

@jwe: I like your proposition in comment #17.
That logic will probably succeed for most use cases.

We could treat `a -= b` (which is a command in Matlab, but an assignment in Octave) as a known incompatibility.
There is syntax that will work the same in both Matlab and in Octave (i.e. using the function syntax): `a ('-=', 'b')`. That is arguably clearer anyway.

Should we use the same logic for Octave's unary appended operators?

Markus Mützel <mmuetzel>
Group administrator
Tue 20 Jul 2021 03:20:54 PM UTC, comment #20: 

The following Octave Forge packages cannot currently be installed on Octave 7 hg id 744bb06fea2a as a result of this change to the parser:


signal (therefore communications)

optim (therefore data-smoothing, econometrics)


Anonymous
Mon 19 Jul 2021 07:05:36 PM UTC, comment #19: 

The '=' symbol is always treated as an operator regardless of spacing.

It may be incorrect for Octave to parse "x ++" as a command at the command line when "x" is already defined as a variable.  We may be missing a check for that.

Also, I didn't verify the actual behavior of the current development version of Octave, but if it is still parsing the full command line before evaluating anything, then input like "x = 1; x ++" all on one line would behave differently from executing those two expressions on separate input lines.

John W. Eaton <jwe>
Group administrator
Mon 19 Jul 2021 06:32:47 PM UTC, comment #18: 

Request: please do continue to allow spaces in "x += 1" and "x -= 1" statements. I use those constructions 493 + 138 == 631 times in my code alone, not counting other people's code.

Question: the following test gives an error in one case but not another. Is that intended?

octave:4> x = y = 0; x = y ++, x, y
x = 0
x = 0
y = 1
octave:5> x = y = 0; x ++
error: x used as variable and later as function


Anonymous
Mon 19 Jul 2021 06:00:56 PM UTC, comment #17: 

The Matlab docs also explain that at the command line, Matlab may check the current workspace to determine whether the first symbol in an expression is a variable.  But in a script or function, command syntax is recognized purely from syntax.  They also explain that when looking at syntax alone, '=' is always recognized as assignment regardless of whitespace but


v1 OPv2  %% command
v1 OP v2 %% expression
v1OPv2   %% expression


We attempt the same, but we have the following extra operators:


 !=
 ++  --
 **
.**  .+   .-
 +=   -=   *=   /=   \\=    ^=   **=   &=   |=
.+=  .-=  .*=  ./=  .\\=   .^=  .**=


I already made the OP= operators always behave as operators regardless of spacing and added the following comment to lex.ll:


// Use the following to handle computed assignment operators
// (+=, -=, etc.) in word list commands in a way that is compatible
// with Matlab.  However, that will also make it impossible to use
// these operators with a space before them:
//
//   x = 1;
//   x+=2;   ## ok
//   x+= 2;  ## ok
//   x +=2;  ## error: invalid use of symbol as both variable and command
//   x += 2; ## error: invalid use of symbol as both variable and command


but maybe that's not the correct behavior anyway.  Instead, would it be OK if we could make Octave behave as follows?


  x-=2   %% -= is an operator
  x-= 2  %% -= is an operator
  x -= 2 %% -= is an operator
  x -=2  %% -=2 is an argument to a command-style function call


The difference with Matlab would still be that "-=" by itself is recognized as an argument to a command-style function call because Matlab doesn't have "-=" as an operator so it recognizes this as "-" followed by another character with no space, so that looks like a command-style argument instead of an operator.  This behavior may break some valid Matlab code that currently works in Octave because users expect "-ANYTHING" to be recognized as a command option.

John W. Eaton <jwe>
Group administrator
Mon 19 Jul 2021 05:39:27 PM UTC, comment #16: 


> The documentation of the increment operators doesn't use spaces. Should we write explicitly that spaces are discouraged/forbidden for the appended increment and decrement operators (unless used in an assignment)?


YES!  I find "c ++" to be a very confusing way to write the increment operator on the variable 'c'.  For clarity alone we should recommend cuddling the operator with the variable name.  And we can mention that there are potential problems with parsing with the other syntax.


Rik <rik5>
Group administrator
Mon 19 Jul 2021 05:05:31 PM UTC, comment #15: 

You are right.
In Matlab R2021a:
On the command line:

>> clear all
>> a=1;
>> b=2;
>> a -b
Error: "a" was previously used as a variable, conflicting with its use here as the name of a function or command.
See "How MATLAB Recognizes Command Syntax" in the MATLAB documentation for details.


With a script:

>> type tst_command_script

a=1;
b=2

a -b
>> tst_command_script
Error: File: tst_command_script.m Line: 4 Column: 1
"a" was previously used as a variable, conflicting with its use here as the name of a function or command.
See "How MATLAB Recognizes Command Syntax" in the MATLAB documentation for details.


With a function:

>> type tst_command_function

function tst_command_function()

a=1;
b=2

a -b

end
>> tst_command_function
Error: File: tst_command_function.m Line: 6 Column: 1
"a" was previously used as a variable, conflicting with its use here as the name of a function or command.
See "How MATLAB Recognizes Command Syntax" in the MATLAB documentation for details.


The current behavior might be the best we can do when it comes to compatibility and syntax extension.
Nevertheless, this change might be breaking some existing code that was written for Octave.
Would it be possible to emit a warning about deprecated syntax in this case and still recognize some operators which are currently treated as arguments?

Independent of this: Should we add a note about this in the NEWS file?

The documentation of the increment operators doesn't use spaces. Should we write explicitly that spaces are discouraged/forbidden for the appended increment and decrement operators (unless used in an assignment)?

Markus Mützel <mmuetzel>
Group administrator
Mon 19 Jul 2021 04:48:23 PM UTC, comment #14: 

I believe that is the way that Matlab parses this syntax because it doesn't have "++" or "--" operators.  When I made the previous change, I made the "OP=" operators special so that they are not recognized as command-style syntax arguments but I missed others.

I believe Matlab command-style parsing depends on whitespace.  For example, "foo - var" (expression) vs. "foo -var" (command), correct?  And the behavior may be different at the command line vs. in a function or script.

So I'm not sure what the best solution is for providing a reasonable degree of compatibility vs. Octave extensions to the Matlab language.


John W. Eaton <jwe>
Group administrator
Mon 19 Jul 2021 03:49:48 PM UTC, comment #13: 

This change might have caused bug #60920.

That bug is about the optim package. But the error might be in the parser/interpreter:

octave:2> which lsqlin
error: parse error near line 118 of file /path/to/optim-1.6.1/lsqlin.m

  n_out: invalid use of symbol as both variable and command

>>>     n_out --;
       ^


It looks like appended increment or decrement operators are interpreted as function arguments.

Markus Mützel <mmuetzel>
Group administrator
Mon 19 Jul 2021 02:53:00 PM UTC, comment #12: 

I made the additional changes described in comment #11 in the following changeset:

http://hg.savannah.gnu.org/hgweb/octave/rev/e2e493712818

OK to close this report now?

John W. Eaton <jwe>
Group administrator
Wed 07 Jul 2021 03:30:44 AM UTC, comment #11: 

I pushed the following change for now:

http://hg.savannah.gnu.org/hgweb/octave/rev/0b01806bb663

As noted in one of the comments I added, it might be possible to store the (expected) variable status of symbols in the symbol_scope object instead of in a separate list of sets.

Also, as noted in the discussion here:  https://octave.discourse.group/t/need-help-understanding-matlab-behavior/1313/12, we might be able to do a better job of parsing command-style function call syntax and detecting variables later used as commands if we always consider an identifier at the beginning of a statement as a possible command when parsing a function and parse it that way regardless of the status of any symbols that appear to be variables. Then, after the file is completely parsed and subfunctions and nested function scopes have been sorted out, walk the parse trees of the functions defined in the file and detect any command-style function calls that are using a symbol name that has previously appeared to be used as a variable. If I have some free time I’ll take a look at doing that.

John W. Eaton <jwe>
Group administrator
Tue 06 Jul 2021 08:30:41 PM UTC, comment #10: 

I'm attaching an updated diff.  It still needs a commit message and some tests.  I'll try to get to that tonight.  I could also use some feedback my latest question here:

https://octave.discourse.group/t/need-help-understanding-matlab-behavior/1313/10


(file #51654)

John W. Eaton <jwe>
Group administrator
Tue 06 Jul 2021 06:33:48 PM UTC, comment #9: 

I think we just need a stack of sets of local variables recognized by the parser instead of just a single set.  I'm looking at it now.  If I can't make a proper change quickly, I'll push the simpler fix.

John W. Eaton <jwe>
Group administrator
Tue 06 Jul 2021 06:21:51 PM UTC, comment #8: 

@jwe: Is it okay to push the patch you attached to this bug report to at least resolve this one issue?  Or do you think that you will get time to re-visit some of the larger issues with variables in different contexts soon and that work will obviate the need for this patch?

Rik <rik5>
Group administrator
Mon 05 Jul 2021 06:49:16 PM UTC, comment #7: 

Thank you. Your patch fixes the issue for me.

Your comment doesn't get displayed entirely, the example with nested functions is:


function main ()
  function nested ()
    warning = 1;
    ...
  end
  warning off all
end


Guillaume <gyom>
Mon 05 Jul 2021 05:28:25 PM UTC, comment #6: 

This error was introduced with changeset 6858992dfadf
because "diary" was being added as a variable name in the expression


  job.diary = ...


because the name was followed by '='.

I think the attached change will help.

While looking at this, I realized that there are some other issues that need to be fixed to properly reset the parser and lexer state when parsing nested functions.  For example, Octave will fail to parse the following even though the "warning" variable inside the nested function should be local.  Some more work needs to be done to properly track the status of identifiers as variables in contexts like this.


function main ()
  function nested ()
    warning = 1;
    ...
  end
  warning off all
end
-verbatim


(file #51648)

John W. Eaton <jwe>
Group administrator
Mon 05 Jul 2021 04:47:23 PM UTC, comment #5: 

Sorry, messed up the verbatim tags. Here it is again:


function varargout = parse_error (varargin)
  job.format = 0;
  format "long g"     %% no parse error
  format "long" "g"   %% this works too
endfunction


Anonymous
Mon 05 Jul 2021 04:46:03 PM UTC, comment #4: 

Turns out it doesn't need the function form diary('off') but simply calling diary "off" with quotes but without parentheses also works:


function varargout = parse_error (varargin)
  job.format = 0;
  format "long g"      %% no parse error here
  format "long" "g"    %% this works too
endfunction

Anonymous
Mon 05 Jul 2021 04:39:09 PM UTC, comment #3: 

This is like a function name being shadowed by a variable name. The problem is when the variable name is from another scope or inside another structure.

The same thing happens with other command-style functions:


function varargout = parse_error (varargin)
  job.axis = 0;
  axis off
endfunction



function varargout = parse_error (varargin)
  job.format = 0;
  format long g
endfunction


Anonymous
Mon 05 Jul 2021 04:31:43 PM UTC, comment #2: 

Possible relations: bug #48031 (fixed) and bug #51988 (open).

Anonymous
Mon 05 Jul 2021 03:33:32 PM UTC, comment #1: 

Confirmed.

If the structure variable is renamed to something besides 'diary' then the code works correctly.  Similarly, if the the code `diary off` is changed to use the functional form of calling


diary ('off')


then the code works.

It would appear that the interpreter is somehow accidentally maintaining state that shows the structure field 'diary' as a top-level variable?

Rik <rik5>
Group administrator
Mon 05 Jul 2021 09:33:15 AM UTC, original submission:  

Using current dev (hg id: b2455f0a8297+), there is a parse error with the following example:


function varargout = parse_error (varargin)

  job.diary = {};

  diary off

endfunction



octave> parse_error
error: parse error near line 5 of file parse_error.m

  syntax error

>>>   diary off
              ^


Guillaume <gyom>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #51654:  mark-as-var-patch-v2.txt added by jwe (8KiB - text/plain)
file #51648:  mark-as-var-patch.txt added by jwe (2KiB - text/plain)

 

Carbon-Copy List
  • -email is unavailable- added by mmuetzel (Posted a comment)
  • -email is unavailable- added by jwe (Updated the item)
  • -email is unavailable- added by rik5 (Posted a comment)
  • -email is unavailable- added by gyom (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 10 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2021-11-23 jwe StatusIn Progress Fixed
        Open/ClosedOpen Closed
    2021-07-21 mmuetzel StatusReady For Test In Progress
    2021-07-19 mmuetzel Dependencies- bugs #60941 is dependent
    2021-07-19 mmuetzel Dependencies- bugs #60920 is dependent
    2021-07-07 jwe StatusConfirmed Ready For Test
    2021-07-06 jwe Attached File- Added mark-as-var-patch-v2.txt, #51654
    2021-07-05 jwe Attached File- Added mark-as-var-patch.txt, #51648
        SummaryParse error error parsing command syntax
    2021-07-05 rik5 StatusNone Confirmed

    Back to the top

    Powered by Savane 3.13-d3ae.
    Corresponding source code