bugGNU Octave - Bugs: bug #62283, max (a, b) produces unintuitive...


bug #62283: max (a, b) produces unintuitive results when mixing integers and floating point values

Submitter:  None
Submitted:  Sun 10 Apr 2022 06:53:18 PM UTC
Category:  Octave Function Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Documentation
Status:  Fixed Assigned to:  None
Originator Name:  Originator Email:  -email is unavailable-
Open/Closed:  * Closed Release:  * dev
Operating System:  * Any Fixed Release:  None
Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup

Jump to the original submission

Tue 28 Jun 2022 01:25:11 AM UTC, comment #12: 

No changes in 3 weeks. Closing as fixed.

Arun Giridhar <arungiridhar>
Group Member
Mon 06 Jun 2022 07:48:25 PM UTC, comment #11: 

That needed a change to the section since it was using promotion/demotion throughout instead of conversion. I pushed out this change to stable: https://hg.savannah.gnu.org/hgweb/octave/rev/2dee06f4635c

Arun Giridhar <arungiridhar>
Group Member
Mon 06 Jun 2022 03:22:37 PM UTC, comment #10: 

I think "conversion" is appropriate.  Type double is the default for calculations in Octave, but whenever it is mixed with another type the rules (inherited from Matlab) require using the more restricted data type and that means values may be changed (such as dropping decimal portion of floating point on conversion to integer).  Conversions:

double OPERATOR single -> single data type
floating point (single or double) OPERATOR integer -> integer data type

Rik <rik5>
Group administrator
Mon 06 Jun 2022 11:18:32 AM UTC, comment #9: 

The usage of these words for describing the behavior in C++ might not apply directly to Octave (e.g., because Octave's variables are usually untyped). But maybe we could try to align as much as possible with how they use these words.
https://en.cppreference.com/w/cpp/language/implicit_conversion (In particular, the sections "Numeric promotions" and "Numeric conversions".)

IIUC, they use "promotion" when an integer value changes to a larger integer type. But only for certain combinations of "source integer type" and "target integer type". Additionally, changing a single precision floating point value to a double precision floating point type seems to be called "promotion" in C++.
IIUC, a "promotion" guarantees that the value doesn't change.

Afaict, the "normal" behavior in Octave is to implicitly change double precision floating point values to single precision floating point type in mixed operations. I guess the latter could be called "demotion". But I don't know if that word is used in that context anywhere else.

When it comes to changing the type between integer types not covered by promotion rules or between integer types and floating point types, they talk about "conversion" (no guarantee that the value won't change).

IIUC, a conversion between floating point and integer types is never called "promotion" by them (and probably should neither be called "demotion" in Octave).

Afaict, there is no distinction between promotion and conversion in Octave. But there are also no guarantees that the value won't change. So, would it be better to talk about "conversions" always? Especially when it comes to values getting converted between integer and floating point types?

Markus Mützel <mmuetzel>
Group administrator
Sun 05 Jun 2022 03:22:18 PM UTC, comment #8: 

I was using the following preexisting text in that section as a guide for my additions:

...where the above operator works with an 8-bit integer and a double
precision value and returns an 8-bit integer value.  Note that the
type is demoted to an 8-bit integer, rather than promoted to a
double precision value as might be expected.  The reason is that
if Octave promoted values in expressions like the above with all
numerical constants would need to be explicitly cast...

From that text, I understood promotion to be changing the data type to one that can accommodate a bigger range, therefore the same as upcasting, and demotion to be the same as downcasting. Is that correct?

Arun Giridhar <arungiridhar>
Group Member
Sun 05 Jun 2022 11:20:15 AM UTC, comment #7: 

I'm not sure the wording in that documentation change is correct.
IIUC, usually integer promotion only refers to promotion of some integer type value to a larger integer type. When a value changes type from integer to floating point, that is usually called "casting".

The "normal" rule in Octave (and Matlab) is to not allow operations between mixed integer type values. Floating point types are implicitly casted to integer types in mixed floating point-integer operations. Double precision floating-point types are implicitly casted to single precision in mixed single-double floating point operations.
In general, I understand the rule is: Double precision floating point numbers are the "default type". If there is an operation involving another type, the user needed to explicitly select that type. So the assumption is that the user would like to keep with the type they explicitly selected (and not switch to the "default" type again and again).
Matlab is a bit stricter than Octave when it comes to that rule: It forbids operations between integer type and single precision floating point numbers. Octave implicitly casts to the integer type in that case (like it would do in mixed double precision floating point-integer operations).

Some functions explicitly cast integer type input to double. Others, automatically promote explicitly to the larger integer type. But those are kind of "exceptions to the rule".

Markus Mützel <mmuetzel>
Group administrator
Sat 04 Jun 2022 09:30:22 PM UTC, comment #6: 

I note that in the course of writing this documentation, I found the following weird inconsistent behavior.

>> a = max (int8 (100), int16 (200)), class (a)
a = 200
ans = int16

>> a = max (int8 (100), double (200)), class (a)
a = 127
ans = int8

So: integers of differing widths are promoted to the wider width, but integer and floating point is demoted to the most binding constraint. Is this done for Matlab compatibility?

Arun Giridhar <arungiridhar>
Group Member
Sat 04 Jun 2022 09:22:19 PM UTC, comment #5: 

I have pushed this documentation change to stable: https://hg.savannah.gnu.org/hgweb/octave/rev/43974344fe19

There seems to be so much function-specific variety that I am not certain I covered all base cases, but I think I got a reasonable diversity of behavior in there.

Marking as Ready For Test (review documentation etc).

Arun Giridhar <arungiridhar>
Group Member
Mon 11 Apr 2022 08:20:22 PM UTC, comment #4: 

See also bug #60784 and bug #60968 for example. Automatic type conversion has always been tricky and subjective.

Arun Giridhar <arungiridhar>
Group Member
Mon 11 Apr 2022 07:13:56 PM UTC, comment #3: 

This issue can manifest itself in many ways.
Maybe an idea to issue a warning with ID (that can be disabled/enabled) in an earlier stage:

warning: double (%g) truncated to (%d) on conversion to type %s

But I guess that opens a box of pandora.

A.R. Burgers <arb>
Mon 11 Apr 2022 06:31:02 PM UTC, comment #2: 

A possible documentation fix is attached for your review.

(file #53076)

Arun Giridhar <arungiridhar>
Group Member
Sun 10 Apr 2022 08:58:12 PM UTC, comment #1: 

Confirmed.  I re-titled the report to be a little clearer.  A agree that the behavior appears strange.  It is, however, explainable and also Matlab-compatible.

I can explain what is happening for this code

a = int8 (100);  b = 200;  max (a, b)

First, Octave/Matlab always returns a value of integer type when a mixed computation of floating point and integer values is used.
 For example,

y = int8 (2) + double (2)
y = 4
class (y)

When Octave calculates

max (int8 (100), double (200))

it respects this rule and returns a result of class int8.

The first part of the calculation is to force the floating point number to be of type int8 but since 200 is greater than the maximum representable value it just returns that max value which is 127

int8 (200)
ans = 127

After that, the rest of the calculation is straightforward

max (int8 (100), int8 (127))
ans = 127

Rik <rik5>
Group administrator
Sun 10 Apr 2022 06:53:18 PM UTC, original submission:  

 a = int8 (100);  b = 200;  max (a, b)

Expected 200, but got 127.

Weird rule for integers. Better to say "cannot call max for mixing int8 and double".

Was not documented in max so this bug report to ask to document this weird result.



(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:

Attached Files
file #53076:  doc.patch added by arungiridhar (1KiB - text/x-patch)


Depends on the following items: None found

Items that depend on this one: None found


Carbon-Copy List
  • -email is unavailable- added by mmuetzel (Posted a comment)
  • -email is unavailable- added by arb (Posted a comment)
  • -email is unavailable- added by arungiridhar (Updated the item)
  • -email is unavailable- added by rik5 (Posted a comment)

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.


    Follow 10 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2022-06-28 arungiridhar StatusReady For Test Fixed
        Open/ClosedOpen Closed
    2022-06-04 arungiridhar StatusConfirmed Ready For Test
    2022-04-11 arungiridhar Attached File- Added doc.patch, #53076
    2022-04-10 rik5 CategoryDocumentation Octave Function
        Item GroupUnexpected Error or Warning Documentation
        StatusNone Confirmed
        Release6.4.0 dev
        Operating SystemMicrosoft Windows Any
        Summarymax (a, b) gives neither a nor b but a random third value max (a, b) produces unintuitive results when mixing integers and floating point values

    Back to the top

    Powered by Savane 3.13-ee12.
    Corresponding source code