I verified jwe's fix this morning in csets 25450, 25451.
Marking bug as fixed and closing report.

Thanks. I'm working on a way to integrate tests for MEX functions into the test suite. I can start by adding tests for these things and bug #51725 and then attempting to fix this issue without undoing the fix for bug #51725.
Once that is done, we'll have a template for MEX function tests and I'm sure a multitude of contributed tests will come rolling in. :)

Note that, in Matlab, structures containing NULL pointers can be produced directly, without a MEXfile.
Indeed, consider the struct x obtained as follows:
and assume that px is an mxArray* to x. Then, mxGetField (px, 0, "a") returns NULL, while mxGetField (px, 1, "a") returns a nonNULL pointer.

Actually, I concluded in comment #12 that NULL was interpreted as [] in Octave. Further testing showed that this was not the behavior in Matlab. In Comment #18 in bug #51725:
It seems that the second paragraph was prescient. In some cases we need NULL to indicate an empty matrix and in other cases it should mean undefined.
The particular case in bug #51725 was the variable plhs which is of type mxArray, but is also very special. Maybe the correct way to handle this is to revert cset 23914 and then for plhs avoid calling mxArray:as_octave_value () if the ptr is NULL? This would introduce a loop over the number of lhs variables, but that number is usually very small.

Comment #12 in bug #51725, by Rik, also says that NULL should be interpreted as [] (0x0).

I guess now is the time to come up with a framework for MEX interface tests. Otherwise we are likely to just continue going around in circles with these kinds of bugs.

This is the changeset that caused the regression:
http://hg.savannah.gnu.org/hgweb/octave/rev/2e4033e1f28f
We can undo that, but then I think we will be reintroducing the problem reported in bug #bug 51725 that resulted in this change.
Or was that change just the wrong thing to do?

The patch solves the problem in the particular case that I initially described, but not from a more general perspective.
Indeed, mxCreateStructArray and mxCreateStructMatrix still create structures initialized with NULL pointers (as they are supposed to, according the Matlab's documentation).
Consider then the following example:
In Matlab this is OK, but in Octave 4.4 it results in the same problem as in my original post.

My change stores [] instead of NULL.
So, I guess you are saying that the previous code was correct to store NULL. Could you check in a MEX file that Matlab actually stores NULL instead of [] in this case?
If so, then it is our conversion to an Octave struct array that is wrong.

> Anyway, I think you mean a NULL argument to mxSetFieldByNumber?
In the title, yes, I meant mxSetFieldByNumber. Can you please change that?
> I'm pretty sure that mxCreateDoubleMatrix (0, 0, mxREAL)
> does not return NULL.
Of course not.
> But maybe Matlab handles NULL specially in this case?
Yes. My understanding is that, when a structure is created by mxCreateStructureArray, or even by struct () in Matlab, the fields are initialized to NULL, and Matlab treats this as "empty".
> Regardless, we should definitely not be creating invalid
> objects.
They are not invalid from Matlab's point of view...
> However, I'm curious to know whether it is documented
> that Matlab accepts NULL here to mean "empty matrix".
Sort of. The documentation of mxGetFieldByNumber says that it "returns NULL in C (0 in Fortran) if passed an invalid argument or if there is no value assigned to the specified field."
https://www.mathworks.com/help/matlab/apiref/mxgetfieldbynumber.html
The possibility of a NULL pointer there is thus explicitely documented. What is not clearly written, however, is the fact that this is interpreted as [] if the field is read (but what else could it be ?).

I pushed the following changeset on stable and merged with default:
http://hg.savannah.gnu.org/hgweb/octave/rev/97e64c23fd07

See also bug #51725.

The end of the post disappeared here because of a botched verbatim tag.
Anyway, I think you mean a NULL argument to mxSetFieldByNumber?
I'm pretty sure that mxCreateDoubleMatrix (0, 0, mxREAL) does not return NULL. But maybe Matlab handles NULL specially in this case?
Regardless, we should definitely not be creating invalid objects.
However, I'm curious to know whether it is documented that Matlab accepts NULL here to mean "empty matrix".

Sorry, the end of my post mysteriously disappeared.
I meant: "like this:"

The following MEXfunction:
returns in Octave 4.4 a "corrupt" structure that Octave cannot display:
The same code works fine in Octave <= 4.2 (which is why I labelled this ticket as "regression") and also in Matlab (and thus is also a "Matlab Compatibility" issue, but one ticket cannot belong to two groups ;)).
The problem disappears if an empty 0x0 matrix is created manually, like this:
