bugGNU Octave - Bugs: bug #63940, `format native-bit` gives wrong...


bug #63940: `format native-bit` gives wrong results

Submitter:  Arun Giridhar <arungiridhar>
Submitted:  Sat 18 Mar 2023 12:19:11 AM UTC
Category:  Octave Function Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Incorrect Result
Status:  In Progress Assigned to:  None
Originator Name:  Open/Closed:  * Open
Release:  * dev Operating System:  * Any
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment (Rich Markup)

( Jump to the original submission )

Mon 20 Mar 2023 12:58:35 AM UTC, comment #6: 

Sorry, wait! It's only been fixed for floating point, but integer inputs still cause an inconsistency between native-bit and native-hex.

Will work on this as I get time but it might be a couple of weeks.

An ideal solution would be if C++ would print directly in bit format like for ios::hex and ios::oct, but it doesn't, requiring hacks like bitset<8>.


Arun Giridhar <arungiridhar>
Project Member
Mon 20 Mar 2023 12:49:21 AM UTC, comment #5: 

I reviewed the committed changeset.  Marking bug as fixed and closing report.

Rik <rik5>
Project Administrator
Sun 19 Mar 2023 12:47:01 AM UTC, comment #4: 

matlab 2022b:

>> format hex
>> uint16(45678)

ans =



>> int16(32001)

ans =



Nicholas Jankowski <nrjank>
Project Member
Sat 18 Mar 2023 08:04:01 PM UTC, comment #3: 

@nrjank: Thanks for checking. Could you also check Matlab for what it does for these?

>> format hex; uint16 (45678)
>> format hex;  int16 (32001)

Octave gives b26e and 7d01 respectively. The equivalent for `bit` matches that of `hex` but `native-bit` does not match `native-hex` for integer inputs and learning Matlab's output will give a target.

I expanded the existing diff into a full patch with more documentation. Since this is addressing an internal inconsistency in Octave, I pushed it to stable here: https://hg.savannah.gnu.org/hgweb/octave/rev/d4d3098a15dc. It's correct for floating point. Based on Matlab's integer behavior in the tests above, it may have to be tuned a bit.

This is correct behavior from unpatched Octave (it matches the documentation for hex, native-hex, and bit):

octave:2> format hex; pi
ans = 400921fb54442d18

octave:3> format native-hex; pi
ans = 182d4454fb210940

octave:4> format bit; pi
ans = 0100000000001001001000011111101101010100010001000010110100011000

This last one is incorrect without the patch (it is inconsistent with Octave's own native-hex):

octave:5> format native-bit; pi
ans = 0001100010110100001000100010101011011111100001001001000000000010

To be consistent with 182d4454fb210940, it should give 0001 1000 0010 1101 0100 0100 0101 0100 1111 1011 0010 0001 0000 1001 0100 0000, but instead it gives 0001 1000 1011 0100 0010 0010 0010 1010 1101 1111 1000 0100 1001 0000 0000 0010 which is equivalent to 18b4222adf849002 in hex, not matching anything.

With the patch it gives the correct behavior:

octave:1> format hex; pi
ans = 400921fb54442d18

octave:2> format native-hex; pi
ans = 182d4454fb210940

octave:3> format bit; pi
ans = 0100000000001001001000011111101101010100010001000010110100011000

octave:4> format native-bit; pi
ans = 0001100000101101010001000101010011111011001000010000100101000000

Arun Giridhar <arungiridhar>
Project Member
Sat 18 Mar 2023 04:34:10 PM UTC, comment #2: 

easy check here.  matlab 2022b:

>> pi

ans =


>> format bit
Error using format
Invalid format option bit.

>> format native-bit
Error using format
Invalid format option native-bit.

the only non-decimal format matlab has is hex. 

>> format hex
>> pi

ans =


based on their description for FORMAT HEX:
"The HEX display corresponds to the internal representation of the value and is not the same as the hexadecimal notation in the C programming language."

I'm sure sometime they'll implement a binary format and it'll conflict with ours :)

Nicholas Jankowski <nrjank>
Project Member
Sat 18 Mar 2023 12:43:26 PM UTC, comment #1: 

The fix turns out to be very simple:

diff -r 48eaf9df5a5b libinterp/corefcn/pr-output.cc
--- a/libinterp/corefcn/pr-output.cc    Mon Mar 13 14:10:11 2023 -0400
+++ b/libinterp/corefcn/pr-output.cc    Sat Mar 18 08:41:33 2023 -0400
@@ -1452,7 +1452,7 @@ pr_any_float (std::ostream& os, const fl
           if (bit_format > 1)
               for (std::size_t i = 0; i < sizeof (T); i++)
-                PRINT_CHAR_BITS_SWAPPED (os, tmp.i[i]);
+                PRINT_CHAR_BITS (os, tmp.i[i]);

Will wait for confirmation against Matlab.

Incidentally, `format hex` and `format native-hex` do the right thing, but `native-bit` is not consistent with `native-hex` without this patch.

Arun Giridhar <arungiridhar>
Project Member
Sat 18 Mar 2023 12:19:11 AM UTC, original submission:  

The documentation for `format` says:

    Print the bit representation of numbers as stored in memory.
    For example, the value of ‘pi’ is


    (shown here in two 32 bit sections for typesetting purposes)
    when printed in native-bit format on a workstation which
    stores 8 byte real values in IEEE format with the least
    significant byte first.

Note: least significant *byte* first, not bit.

But running `format native-bit; pi` gives:


which is not the same as the help text. In fact it's the exact reverse of what the help text says should be printed.

To compare, `format bit` prints consistently with the most significant *bit* first and it gives this:


which is identical to the help text for `native-bit`, and therefore inconsistent with a little-endian byte order in a word. (The results were on an x86_64, little-endian architecture).

I verified that the `format bit` output is correct for what it claims to represent in MSB-first order: it represents

(1 + sum(2 .^ -find("1001001000011111101101010100010001000010110100011000" == "1"))) * 2 ^ 1

in IEEE 754 format, and evaluates to `pi` as it should.

So it looks like the documentation for `native-bit` is wrong, in that its example for pi will not show up on little-endian machines but only big-endian ones like SPARC.

It looks like the code for `format native-bit` is also wrong, because it should only be returning the least significant byte first and the most significant byte last, but the order of bits inside a byte should not be affected by that order of bytes in a word. (Recall the Unicode byte order mark FFFE shows up as FEFF when switching endianness, not as 7FFF.)

What does Matlab do for `format bit; pi` and `format native-bit; pi`?

Arun Giridhar <arungiridhar>
Project Member


(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:

No files currently attached


Depends on the following items: None found

Items that depend on this one: None found


Carbon-Copy List
  • -email is unavailable- added by rik5 (Posted a comment)
  • -email is unavailable- added by nrjank (Posted a comment)
  • -email is unavailable- added by arungiridhar (Submitted the item)

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only project members can vote.





    Follow 6 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2023-03-20 arungiridhar StatusFixed In Progress
        Open/ClosedClosed Open
    2023-03-20 rik5 StatusIn Progress Fixed
        Open/ClosedOpen Closed
    2023-03-18 arungiridhar StatusPatch Submitted In Progress
    2023-03-18 arungiridhar StatusNone Patch Submitted

    Back to the top

    Powered by Savane 3.11