bugGNU Octave - Bugs: bug #63940, `format native-bit` gives wrong...


bug #63940: `format native-bit` gives wrong results

Submitter:  Arun Giridhar <arungiridhar>
Submitted:  Sat 18 Mar 2023 12:19:11 AM UTC
Category:  Octave Function Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Incorrect Result
Status:  Fixed Assigned to:  None
Originator Name:  Open/Closed:  * Closed
Release:  * dev Operating System:  * Any
Fixed Release:  8.2.0 Planned Release:  8.2.0
* Mandatory Fields

Add a New Comment Rich Markup

Jump to the original submission

Thu 23 Mar 2023 03:48:05 PM UTC, comment #7: 

It was an easier change than I had thought. The previous edit needed to be done in two places, once for integers and once for floating point.

Pushed to stable at https://hg.savannah.gnu.org/hgweb/octave/rev/dd23a26b7294

Marking as fixed and closing. If anyone sees problems with format, it can be reopened.

Arun Giridhar <arungiridhar>
Group Member
Mon 20 Mar 2023 12:58:35 AM UTC, comment #6: 

Sorry, wait! It's only been fixed for floating point, but integer inputs still cause an inconsistency between native-bit and native-hex.

Will work on this as I get time but it might be a couple of weeks.

An ideal solution would be if C++ would print directly in bit format like for ios::hex and ios::oct, but it doesn't, requiring hacks like bitset<8>.


Arun Giridhar <arungiridhar>
Group Member
Mon 20 Mar 2023 12:49:21 AM UTC, comment #5: 

I reviewed the committed changeset.  Marking bug as fixed and closing report.

Rik <rik5>
Group administrator
Sun 19 Mar 2023 12:47:01 AM UTC, comment #4: 

matlab 2022b:

>> format hex
>> uint16(45678)

ans =



>> int16(32001)

ans =



Nicholas Jankowski <nrjank>
Group Member
Sat 18 Mar 2023 08:04:01 PM UTC, comment #3: 

@nrjank: Thanks for checking. Could you also check Matlab for what it does for these?

>> format hex; uint16 (45678)
>> format hex;  int16 (32001)

Octave gives b26e and 7d01 respectively. The equivalent for `bit` matches that of `hex` but `native-bit` does not match `native-hex` for integer inputs and learning Matlab's output will give a target.

I expanded the existing diff into a full patch with more documentation. Since this is addressing an internal inconsistency in Octave, I pushed it to stable here: https://hg.savannah.gnu.org/hgweb/octave/rev/d4d3098a15dc. It's correct for floating point. Based on Matlab's integer behavior in the tests above, it may have to be tuned a bit.

This is correct behavior from unpatched Octave (it matches the documentation for hex, native-hex, and bit):

octave:2> format hex; pi
ans = 400921fb54442d18

octave:3> format native-hex; pi
ans = 182d4454fb210940

octave:4> format bit; pi
ans = 0100000000001001001000011111101101010100010001000010110100011000

This last one is incorrect without the patch (it is inconsistent with Octave's own native-hex):

octave:5> format native-bit; pi
ans = 0001100010110100001000100010101011011111100001001001000000000010

To be consistent with 182d4454fb210940, it should give 0001 1000 0010 1101 0100 0100 0101 0100 1111 1011 0010 0001 0000 1001 0100 0000, but instead it gives 0001 1000 1011 0100 0010 0010 0010 1010 1101 1111 1000 0100 1001 0000 0000 0010 which is equivalent to 18b4222adf849002 in hex, not matching anything.

With the patch it gives the correct behavior:

octave:1> format hex; pi
ans = 400921fb54442d18

octave:2> format native-hex; pi
ans = 182d4454fb210940

octave:3> format bit; pi
ans = 0100000000001001001000011111101101010100010001000010110100011000

octave:4> format native-bit; pi
ans = 0001100000101101010001000101010011111011001000010000100101000000

Arun Giridhar <arungiridhar>
Group Member
Sat 18 Mar 2023 04:34:10 PM UTC, comment #2: 

easy check here.  matlab 2022b:

>> pi

ans =


>> format bit
Error using format
Invalid format option bit.

>> format native-bit
Error using format
Invalid format option native-bit.

the only non-decimal format matlab has is hex. 

>> format hex
>> pi

ans =


based on their description for FORMAT HEX:
"The HEX display corresponds to the internal representation of the value and is not the same as the hexadecimal notation in the C programming language."

I'm sure sometime they'll implement a binary format and it'll conflict with ours :)

Nicholas Jankowski <nrjank>
Group Member
Sat 18 Mar 2023 12:43:26 PM UTC, comment #1: 

The fix turns out to be very simple:

diff -r 48eaf9df5a5b libinterp/corefcn/pr-output.cc
--- a/libinterp/corefcn/pr-output.cc    Mon Mar 13 14:10:11 2023 -0400
+++ b/libinterp/corefcn/pr-output.cc    Sat Mar 18 08:41:33 2023 -0400
@@ -1452,7 +1452,7 @@ pr_any_float (std::ostream& os, const fl
           if (bit_format > 1)
               for (std::size_t i = 0; i < sizeof (T); i++)
-                PRINT_CHAR_BITS_SWAPPED (os, tmp.i[i]);
+                PRINT_CHAR_BITS (os, tmp.i[i]);

Will wait for confirmation against Matlab.

Incidentally, `format hex` and `format native-hex` do the right thing, but `native-bit` is not consistent with `native-hex` without this patch.

Arun Giridhar <arungiridhar>
Group Member
Sat 18 Mar 2023 12:19:11 AM UTC, original submission:  

The documentation for `format` says:

    Print the bit representation of numbers as stored in memory.
    For example, the value of ‘pi’ is


    (shown here in two 32 bit sections for typesetting purposes)
    when printed in native-bit format on a workstation which
    stores 8 byte real values in IEEE format with the least
    significant byte first.

Note: least significant *byte* first, not bit.

But running `format native-bit; pi` gives:


which is not the same as the help text. In fact it's the exact reverse of what the help text says should be printed.

To compare, `format bit` prints consistently with the most significant *bit* first and it gives this:


which is identical to the help text for `native-bit`, and therefore inconsistent with a little-endian byte order in a word. (The results were on an x86_64, little-endian architecture).

I verified that the `format bit` output is correct for what it claims to represent in MSB-first order: it represents

(1 + sum(2 .^ -find("1001001000011111101101010100010001000010110100011000" == "1"))) * 2 ^ 1

in IEEE 754 format, and evaluates to `pi` as it should.

So it looks like the documentation for `native-bit` is wrong, in that its example for pi will not show up on little-endian machines but only big-endian ones like SPARC.

It looks like the code for `format native-bit` is also wrong, because it should only be returning the least significant byte first and the most significant byte last, but the order of bits inside a byte should not be affected by that order of bytes in a word. (Recall the Unicode byte order mark FFFE shows up as FEFF when switching endianness, not as 7FFF.)

What does Matlab do for `format bit; pi` and `format native-bit; pi`?

Arun Giridhar <arungiridhar>
Group Member


(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:

No files currently attached


Depends on the following items: None found

Items that depend on this one: None found


Carbon-Copy List
  • -email is unavailable- added by rik5 (Posted a comment)
  • -email is unavailable- added by nrjank (Posted a comment)
  • -email is unavailable- added by arungiridhar (Submitted the item)

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only project members can vote.


    Follow 10 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2023-03-23 arungiridhar StatusIn Progress Fixed
        Open/ClosedOpen Closed
        Fixed ReleaseNone 8.2.0
        Planned ReleaseNone 8.2.0
    2023-03-20 arungiridhar StatusFixed In Progress
        Open/ClosedClosed Open
    2023-03-20 rik5 StatusIn Progress Fixed
        Open/ClosedOpen Closed
    2023-03-18 arungiridhar StatusPatch Submitted In Progress
    2023-03-18 arungiridhar StatusNone Patch Submitted

    Back to the top

    Powered by Savane 3.12