Thanks for checking.

In hindsight, I should probably opened a new report for this.

I pushed the following change which I hope does the "right thing"™:
https://hg.savannah.gnu.org/hgweb/octave/rev/9afc383bb60a

Somebody who better understands the parser should probably double-check though.

Thanks for catching that oversight.  I fixed it in this changeset http://hg.savannah.gnu.org/hgweb/octave/rev/9e0421bd8bad.

I pushed a documentation change and NEWS entry about the no longer allowed whitespaces between increment/decrement operator and the variable they affect here:
https://hg.savannah.gnu.org/hgweb/octave/rev/7ecd07721cca

Is there anything else to do here?  The last thing mentioned was that we would start deprecating '**' and '.**' exponent operators.  That isn't specific to this bug report so this one could probably be closed.

Rik: until recent changes, when executing code at the command line, Octave checked whether a symbol was defined as a variable and if so, did not attempt to parse a statement as a command-style function call.  I believe that is also what Matlab used to do.  But now, for consistency in behavior, Matlab uses the same syntax-only rules to parse statements.  It doesn't consult the current workspace at parse time.  The "used as variable and later as function" message happens after parsing is already done, at the time the code is executed.

I think the only question left is whether to handle our extended set of operators (++, +=, etc.) specially or in a fully Matlab-compatible way when parsing command syntax.  Since Matlab doesn't have += (for example), I believe that an expression like

foo +=

is parsed as a command-style function call regardless of what follows the +=.  If we do the same for Octave, then it is impossible to use += as an operator unless the expression is written with no space after the identifier:

foo+=something    ## OP=
foo+= something   ## OP=
foo +=something   ## command-style function call
foo += something  ## command-style function call

That seems bad to me, so I left the special treatment of these operators in Octave so that the only one that causes command-style function parsing is

foo +=something

similar to the way that both Octave and Matlab handle other binary operators (i.e., only the "foo +something" is handled as command-style function syntax).

Since ++ is a unary operator and + by itself is both a binary and prefix unary operator, the rules are a little different.  In Matlab, the parsing of expressions involving repeated + characters is a bit weird:

foo ++            %% command-style function call
foo ++ something  %% command-style function call
foo++something    %% expression: foo + (+something)
foo++ something   %% expression: foo + (+something)
foo ++something   %% command-style function call

See also

https://octave.discourse.group/t/parsing-command-style-function-call-syntax/1414/4

where I posted a test function to verify current behavior of Matlab and Octave.  Note that in the results that Markus posted for Matlab, the "OK" doesn't mean that Octave and Matlab behavior is the same, just that we know what the behaviors are.  Looking at the function itself shows that there are currently differences between  Octave and Matlab for the following operators:

@
\
~
.+
.-
.**
!=
**
++
--

all the OP= operators.

It would probably be fairly painless to phase out and eventually remove

.+
.-
.**
**

We are already eliminating the problem with backslash as a continuation character outside of character strings.  I think we can make the behavior of @ and ~ compatible with Matlab.  Changing != (and !) to be compatible would be very unpleasant.

Since ++ and -- are not binary operators anyway, we can get closer to the Matlab behavior if we reject them as two-character tokens in some cases depending on the surrounding context, return just '+' (or '-') from the lexer, and continue parsing beginning with the next '+' (or '-') character from the input stream.  That should provide compatibility with Matlab by parsing expressions like "foo++something" as "foo + (+something)".

I should clarify that "x ++ + 3" evaluates to 3 when x is originally 0. I neglected to mention the initial value of x. In general it evaluates to x+3 overall, and increments x as a side-effect. This behavior is actually consistent with C++ and Java, so might not be wrong though it is ill-advised to write such an expression in production code.

I request clarification. The statement "x ++" causes an error but the command "y = x ++" does not, which evaluates to "y = x; x += 1;". Nor does the admittedly pathological statement "x ++ + 3" cause an error, which evaluates to 3. That is, "x ++" with the space only causes an error when it occurs by itself, not when embedded in a bigger statement. Is this behavior intended?

Imho, we don't need to aim for full syntax compatibility for the operators that Octave has but Matlab does not have.

From a user point of view, it is possible to write code that is compatible with Octave and with Matlab without too much effort at all: Just avoid command syntax if the arguments look like operators.

BTW, this trouble all started when I was working on handling the new "arguments" block syntax and I decided to try to eliminate some duplicated information in the parser and interpreter.  Instead of storing lists of pending variables (function parameters and identifiers found on the LHS of assignment operators) in the lexer and storing identifiers in the symbol table, I thought it would be best to just tag the symbols in the symbol table as (expected) variables based on syntax alone.  A mistake in that change caused the bug reported here.  (The mistake was entering "diary" as a variable when it appeared in an expression like "job.diary = ...", even though "diary" by itself here isn't a variable.)

Now I think I'm just confused about what we should be doing here, because I just made a change so that we are once again treating

a=1;
b=2;
a -b

at the command line differently from what happens in a function or script because my change made Octave look at the symbol table to decide whether "a" is a variable when parsing "a -b", and if it is, to not handle that statement as a command.  But according to comment #15, current versions of Matlab are no longer doing that?  I think it is better if there is not special treatment at the command line, so I undid that part of my previous change, here:

I also think it is best if we can parse the command-style syntax completely based on syntax, so I would prefer to stick with that change.  But it does present some compatibility problems.  If we use syntax alone and opt for full Matlab compatibility, then none  of the OP= operators can work properly with simple identifiers at the beginning of an expression because Matlab will always parse statements like "foo -= bar" as commands (please correct me if that's not the case).

That doesn't seem to be working in Matlab R2021a either:

>> type +a/b

function b(arg1, arg2)
disp(arg1);
disp(arg2);
end
>> a.b('foo', 'bar')
foo
bar
>> a.b foo bar
 a.b foo bar
     ↑
Invalid expression. Check for missing multiplication operator, missing or unbalanced delimiters, or other syntax error. To construct matrices, use brackets instead of parentheses.

I fixed the one example in core octave in _strip_html_tags_.m.  See http://hg.savannah.gnu.org/hgweb/octave/rev/ee84485289ce.

I used grep to look for ' --' and ' ++' sequences in the scripts directory and that was the only one I found.

Re comment #24: That change only affects the .m file parser. It shouldn't affect C or C++ code.

Thanks.
That might be the best way forward.

With Octave 6, `x --` was interpreted as decrement operation.
IIUC, the same syntax will be interpreted as command syntax in Octave 7. This might break code that currently uses something like that.
I gather there is no easy way to deprecate that syntax for a few versions before we finally change its meaning.

IIUC, syntax like that can be fixed easily by removing the whitespace (`x--`) or by forcing it to an expression (e.g. with `(x --)`). That should have the same meaning in Octave 6 and in Octave 7.

Is there a way we can reach out to users and developers (e.g. of Octave Forge packages) to give them some head start of that upcoming change?

@jwe: I like your proposition in comment #17.
That logic will probably succeed for most use cases.

We could treat `a -= b` (which is a command in Matlab, but an assignment in Octave) as a known incompatibility.
There is syntax that will work the same in both Matlab and in Octave (i.e. using the function syntax): `a ('-=', 'b')`. That is arguably clearer anyway.

Should we use the same logic for Octave's unary appended operators?

The '=' symbol is always treated as an operator regardless of spacing.

It may be incorrect for Octave to parse "x ++" as a command at the command line when "x" is already defined as a variable.  We may be missing a check for that.

Also, I didn't verify the actual behavior of the current development version of Octave, but if it is still parsing the full command line before evaluating anything, then input like "x = 1; x ++" all on one line would behave differently from executing those two expressions on separate input lines.

The Matlab docs also explain that at the command line, Matlab may check the current workspace to determine whether the first symbol in an expression is a variable.  But in a script or function, command syntax is recognized purely from syntax.  They also explain that when looking at syntax alone, '=' is always recognized as assignment regardless of whitespace but

v1 OPv2  %% command
v1 OP v2 %% expression
v1OPv2   %% expression

We attempt the same, but we have the following extra operators:

 !=
 ++  --
 **
.**  .+   .-
 +=   -=   *=   /=   \\=    ^=   **=   &=   |=
.+=  .-=  .*=  ./=  .\\=   .^=  .**=

I already made the OP= operators always behave as operators regardless of spacing and added the following comment to lex.ll:

// Use the following to handle computed assignment operators
// (+=, -=, etc.) in word list commands in a way that is compatible
// with Matlab.  However, that will also make it impossible to use
// these operators with a space before them:
//
//   x = 1;
//   x+=2;   ## ok
//   x+= 2;  ## ok
//   x +=2;  ## error: invalid use of symbol as both variable and command
//   x += 2; ## error: invalid use of symbol as both variable and command

but maybe that's not the correct behavior anyway.  Instead, would it be OK if we could make Octave behave as follows?

  x-=2   %% -= is an operator
  x-= 2  %% -= is an operator
  x -= 2 %% -= is an operator
  x -=2  %% -=2 is an argument to a command-style function call

The difference with Matlab would still be that "-=" by itself is recognized as an argument to a command-style function call because Matlab doesn't have "-=" as an operator so it recognizes this as "-" followed by another character with no space, so that looks like a command-style argument instead of an operator.  This behavior may break some valid Matlab code that currently works in Octave because users expect "-ANYTHING" to be recognized as a command option.

You are right.
In Matlab R2021a:
On the command line:

>> clear all
>> a=1;
>> b=2;
>> a -b
Error: "a" was previously used as a variable, conflicting with its use here as the name of a function or command.
See "How MATLAB Recognizes Command Syntax" in the MATLAB documentation for details.

With a script:

>> type tst_command_script

a=1;
b=2

a -b
>> tst_command_script
Error: File: tst_command_script.m Line: 4 Column: 1
"a" was previously used as a variable, conflicting with its use here as the name of a function or command.
See "How MATLAB Recognizes Command Syntax" in the MATLAB documentation for details.

With a function:

>> type tst_command_function

function tst_command_function()

a=1;
b=2

a -b

end
>> tst_command_function
Error: File: tst_command_function.m Line: 6 Column: 1
"a" was previously used as a variable, conflicting with its use here as the name of a function or command.
See "How MATLAB Recognizes Command Syntax" in the MATLAB documentation for details.

The current behavior might be the best we can do when it comes to compatibility and syntax extension.
Nevertheless, this change might be breaking some existing code that was written for Octave.
Would it be possible to emit a warning about deprecated syntax in this case and still recognize some operators which are currently treated as arguments?

Independent of this: Should we add a note about this in the NEWS file?

The documentation of the increment operators doesn't use spaces. Should we write explicitly that spaces are discouraged/forbidden for the appended increment and decrement operators (unless used in an assignment)?

This change might have caused bug #60920.

That bug is about the optim package. But the error might be in the parser/interpreter:

octave:2> which lsqlin
error: parse error near line 118 of file /path/to/optim-1.6.1/lsqlin.m

  n_out: invalid use of symbol as both variable and command

>>>     n_out --;
       ^

It looks like appended increment or decrement operators are interpreted as function arguments.

I pushed the following change for now:

http://hg.savannah.gnu.org/hgweb/octave/rev/0b01806bb663

As noted in one of the comments I added, it might be possible to store the (expected) variable status of symbols in the symbol_scope object instead of in a separate list of sets.

Also, as noted in the discussion here:  https://octave.discourse.group/t/need-help-understanding-matlab-behavior/1313/12, we might be able to do a better job of parsing command-style function call syntax and detecting variables later used as commands if we always consider an identifier at the beginning of a statement as a possible command when parsing a function and parse it that way regardless of the status of any symbols that appear to be variables. Then, after the file is completely parsed and subfunctions and nested function scopes have been sorted out, walk the parse trees of the functions defined in the file and detect any command-style function calls that are using a symbol name that has previously appeared to be used as a variable. If I have some free time I’ll take a look at doing that.

I think we just need a stack of sets of local variables recognized by the parser instead of just a single set.  I'm looking at it now.  If I can't make a proper change quickly, I'll push the simpler fix.

Thank you. Your patch fixes the issue for me.

Your comment doesn't get displayed entirely, the example with nested functions is:

function main ()
  function nested ()
    warning = 1;
    ...
  end
  warning off all
end

Sorry, messed up the verbatim tags. Here it is again:

function varargout = parse_error (varargin)
  job.format = 0;
  format "long g"     %% no parse error
  format "long" "g"   %% this works too
endfunction

This is like a function name being shadowed by a variable name. The problem is when the variable name is from another scope or inside another structure.

The same thing happens with other command-style functions:

function varargout = parse_error (varargin)
  job.axis = 0;
  axis off
endfunction

function varargout = parse_error (varargin)
  job.format = 0;
  format long g
endfunction

Confirmed.

If the structure variable is renamed to something besides 'diary' then the code works correctly.  Similarly, if the the code `diary off` is changed to use the functional form of calling

diary ('off')

then the code works.

It would appear that the interpreter is somehow accidentally maintaining state that shows the structure field 'diary' as a top-level variable?