Thanks.  I'm working on a way to integrate tests for MEX functions into the test suite.  I can start by adding tests for these things and bug #51725 and then attempting to fix this issue without undoing the fix for bug #51725.

Once that is done, we'll have a template for MEX function tests and I'm sure a multitude of contributed tests will come rolling in.  :-)

Actually, I concluded in comment #12 that NULL was interpreted as [] in Octave.  Further testing showed that this was not the behavior in Matlab.  In Comment #18 in bug #51725:

Yes. NULL values should be converted to undefined, rather than to empty matrix.

After that change it might be necessary to check whether some of the MEX routines have been relying on that behavior to actually return an empty matrix.

It seems that the second paragraph was prescient.  In some cases we need NULL to indicate an empty matrix and in other cases it should mean undefined.

The particular case in bug #51725 was the variable plhs which is of type mxArray, but is also very special.  Maybe the correct way to handle this is to revert cset 23914 and then for plhs avoid calling mxArray:as_octave_value () if the ptr is NULL?  This would introduce a loop over the number of lhs variables, but that number is usually very small.

I guess now is the time to come up with a framework for MEX interface tests.  Otherwise we are likely to just continue going around in circles with these kinds of bugs.

The patch solves the problem in the particular case that I initially described, but not from a more general perspective.

Indeed, mxCreateStructArray and mxCreateStructMatrix still create structures initialized with NULL pointers (as they are supposed to, according the Matlab's documentation).

Consider then the following example:

#include "mex.h"

static const char* field_names[] = {"dudule"};

void mexFunction (int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[])
{
  if (nlhs > 1)   /* Check number of output arguments */
    mexErrMsgTxt ("Too many output arguments.");

  plhs[0] = mxCreateStructMatrix (1, 1, 1, field_names);
}

In Matlab this is OK, but in Octave 4.4 it results in the same problem as in my original post.

> Anyway, I think you mean a NULL argument to mxSetFieldByNumber?

In the title, yes, I meant mxSetFieldByNumber.  Can you please change that?

> I'm pretty sure that mxCreateDoubleMatrix (0, 0, mxREAL)
> does not return NULL.

Of course not.

> But maybe Matlab handles NULL specially in this case?

Yes.  My understanding is that, when a structure is created by mxCreateStructureArray, or even by struct () in Matlab, the fields are initialized to NULL, and Matlab treats this as "empty".

> Regardless, we should definitely not be creating invalid
> objects.

They are not invalid from Matlab's point of view...

> However, I'm curious to know whether it is documented
> that Matlab accepts NULL here to mean "empty matrix".

Sort of.  The documentation of mxGetFieldByNumber says that it "returns NULL in C (0 in Fortran) if passed an invalid argument or if there is no value assigned to the specified field."

https://www.mathworks.com/help/matlab/apiref/mxgetfieldbynumber.html

The possibility of a NULL pointer there is thus explicitely documented.  What is not clearly written, however, is the fact that this is interpreted as [] if the field is read (but what else could it be ?).

See also bug #51725.

Sorry, the end of my post mysteriously disappeared.

I meant: "like this:"

mxSetFieldByNumber (plhs[0], 0, 0, mxCreateDoubleMatrix (0, 0, mxREAL));