Michael L said:
So the upshot: it is desirable to have a welldefined total order also on complex numbers, even if you are not interested in the specific ordering.
========================
This appears to be the best solution. The lexicographic ordering
is fairly "natural" and efficient. Other orderings seem to be
"special cases." It will deserve a clear explanation in the Manual.
In any case, users who have a specific need to "sort" complex numbers
will likely need to choose an ordering that meets their requirements,
and/or preprocess the data.

It's just that the complex numbers are not an ordered field, that is, there is no total order that is compatible with how the natural ordering relation on the real numbers behaves under addition and multiplication. However, as with any set you can also assign a total order to the complex numbers. As I see it, the most intuitive one would be the lexicographic order that is implied by understanding the complex numbers as the Cartesian product R x R, with the two added advantages that in this case the real numbers as R x {0} would then be orderembedded in the complex numbers, and that the numerical effort of a complex comparison would be only insignificantly larger than a real comparison, compared to the present case where two abs() have to be computed for every comparison (and sometimes even two atan2()s).
And if you error out for every complex comparison, you break unique() and thus union(), setdiff() and so on as they are now and need to emulate their behaviour by an explicit unique([real(a)(:) imag(a)(:),"rows") followed by reshaping, which is significantly less efficient and more memoryconsuming. So the upshot: it is desirable to have a welldefined total order also on complex numbers, even if you are not interested in the specific ordering.

Mathematically, the complex numbers are not an ordered set. So formally, any attempt to sort them should produce an error.
However, there are subsets of the complex numbers in which the attempt to produce an order makes some sense. For example, in the case of a vector which results from calculations where the imaginary parts formally would all be zero but some may have accidentally a very small value caused by numerical inacurrancy.
So what about throwing just a warning in cases where all available sort methods would produce the same order but an error when there are different results for the possible methods and no method has been explicitely defined? Same for max and min.

My preference would even be an outright error. I mean, I'd hope there is no code in any packages that are expecting some type of behavior for the order operators on complex numbers or vectors. It's not very difficult to explicitly indicate
or for vectors

I would be satisfied with a warning, on by default. As I mentioned in bug #52919, the way I initially came upon this problem was that it resulted in incorrect behaviour of code of mine, because I used vector comparisons, where sometimes the vector had (a very small number of) complex entries, just as the original reporter of the present bug describes. A warning would have made me immediately aware of the problem.
I can think of only one legitimate use case of comparing complex values, to wit, the function unique. But actually, in this case the total ordering is only necessary for the sort that goes on in the background, and any total order would do. Would there be some way to selectively disable the warning in a sort (or sortrows) that is called from unique? Because it is obviously perfectly valid to request the unique values of a complex vector. All other uses of sort, max, bare < and so on should in my eyes rather explicitly form and compare the absolute values, angles, real parts or whatever.
So actually there is a further argument for changing the ordering on complex numbers to lexicographic ordering: It has significantly higher performance (see bug #53012). So this would allow unique to run much faster on complex vectors than is possible with the present choice without using any additional memory.

I agree, too. A warning when try to order complex values seems the right thing.

That seems right. Unless the programmer has explicitly told Octave which sorting method to use, emit a warning. This is too difficult a decision to leave up to some heuristic which, no matter how it chooses, it going to get it wrong for some users.

So, we should warn in all cases when attempting to order complex values unless explicitly told to use a particular method of ordering.

Yes, I think it was the sort function that now has an option to sort by real value or by abs value, but not max and min.
So if we do become Matlab compatible, the next bug reports will be
Think of a strange scenario where x < y is true, but max([x y]) returns x.

I guess this is the bug that will not die.
Even though this will make the comparison used in the <, <=, >=, and > different from the behavior of min and max, maybe we should just finally give in and submit to the way that Matlab works.
How about also a warning (you may disable if you wish) when using <, <=, >=, or > to compare complex values?

Yes, I have described these issues already at some length in bug #52919 and bug #53013. I would also be in favour of changing the ordering relation on complex numbers to lexicographic, which would solve all the mentioned problems of counterintuitive behaviour and additionally bring about matlab compatibility.

QUIZ: Think of a strange scenario where the result of (x<1)(1) depends on the value of x(2) (!)
Yes, there's such an a possible:
x=[2 1] and x = [2 i] return different answers for (x<1)(1).
That's because octave switches to abs() if it encounters complex numbers.
The problems with this, in my opinion, are:
(1) Strange logic above of result(1) depending on x(2).
(2) A strange discontinuity in y<1 between y = 2 and x = 2 + eps i
(3) Matlab, see below.
(4) As it stands, < is therefore useless and unreliable when using complex numbers. You cannot consistently rely on it to use abs, since you never know when x will be interpreted as real.
A more consistent approach seems to be to use Re() instead of Abs() when comparing complex numbers. Instead of any strange behaviors as above, that transitions smoothly when x becomes real.
Finally, matlab seems to use the behavior I recommended: https://www.mathworks.com/help/matlab/ref/lt.html#bt297ps5
