bugGNU Octave - Bugs: bug #59830, Architecture-dependent results for...

 
 

bug #59830: Architecture-dependent results for arithmetic operations involving NA

Submitted by:  Rafael Laboissiere <rlaboiss>
Submitted on:  Tue 05 Jan 2021 10:10:12 PM UTC  
 
Category:  Interpreter Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Incorrect Result
Status:  Need Info Assigned to:  None
Originator Name:  Open/Closed:  Open
Release:  6.1.0 Operating System:  GNU/Linux

Add a New Comment (Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

( Jump to the original submission)

Wed 20 Jan 2021 08:08:35 AM UTC, comment #17: 

I tried it (using the Debian package for octave) and the configuration fails with the following error message:

configure:6770: checking whether the C compiler works
configure:6792: gcc -g -O2 -fdebug-prefix-map=/home/rafael/octave-6.1.1~hg.2020.12.27=. -fstack-protector-strong -Wformat -Werror=format-security -mnan=2008 -Wdate-time -D_FORTIFY_SOURCE=2 -Wl,-z,relro conftest.c  >&5
/usr/bin/ld: /tmp/cc8LGpyx.o: linking -mnan=2008 module with previous -mnan=legacy modules
/usr/bin/ld: failed to merge target specific data of file /tmp/cc8LGpyx.o
collect2: error: ld returned 1 exit status
configure:6796: $? = 1
configure:6834: result: no
configure: failed program was:
| /* confdefs.h */
| #define PACKAGE_NAME "GNU Octave"
| #define PACKAGE_TARNAME "octave"
| #define PACKAGE_VERSION "6.1.1~hg.2020.12.27-1"
| #define PACKAGE_STRING "GNU Octave 6.1.1~hg.2020.12.27-1"
| #define PACKAGE_BUGREPORT "https://octave.org/bugs.html"
| #define PACKAGE_URL "https://www.gnu.org/software/octave/"
| #define PACKAGE "octave"
| #define VERSION "6.1.1~hg.2020.12.27-1"
| #define OCTAVE_SOURCE 1
| /* end confdefs.h.  */
|
| int
| main ()
| {
|
|   ;
|   return 0;
| }
configure:6839: error: in `/home/rafael/octave-6.1.1~hg.2020.12.27':
configure:6841: error: C compiler cannot create executables

Rafael Laboissiere <rlaboiss>
Tue 19 Jan 2021 05:36:25 PM UTC, comment #16: 

Hmm, it could get to be time consuming to support ARM if there are a lot of instances where this is going to diverge from all the other processor architectures.

What happens if you undo my patch, add '-mnan=2008' to your CXXFLAGS, CFLAGS, and FFLAGS variables, and re-compile Octave from scratch?  Do operations with NA work correctly?  Does 'make check' pass?

Rik <rik5>
Project Administrator
Tue 19 Jan 2021 08:54:22 AM UTC, comment #15: 

comment #13:

> If so, this isn't that hard a fix.  There will need to be a configure test written to determine whether Octave is compiling on a MIPS architecture and then there will need to be an #ifdef in lo-ieee.h to choose between two different definitions of LO_IEEE_NA_HW.


We should also ensure the interoperability between the different architectures.  For instance, if we run the following code on a amd64 system:

octave:1> x = NA;
octave:2> save -binary x.dat x

I get now the following on mips64el with the patched Octave, when trying to load the file saved on amd64:

octave:1> load x.dat
octave:2> x
x = NaN

Rafael Laboissiere <rlaboiss>
Tue 19 Jan 2021 07:31:06 AM UTC, comment #14: 

comment #13:

> It seems that re-defining Octave NA to be a quiet signaling NaN on MIPS architectures will resolve this issue.  To be absolutely certain, could you use this custom-built version of Octave and try running the BIST tests for the level-set package and see if they pass?


I did the required test. I compiled Octave on a mips64el system with your LO_IEEE_NA_HW patch and used that version to run the 18 BISTs in function so_step_armijo of the level-set package, all of which passed.

Rafael Laboissiere <rlaboiss>
Tue 12 Jan 2021 04:58:24 PM UTC, comment #13: 

Apparently my last posting (comment #12) was truncated.

It seems that re-defining Octave NA to be a quiet signaling NaN on MIPS architectures will resolve this issue.  To be absolutely certain, could you use this custom-built version of Octave and try running the BIST tests for the level-set package and see if they pass?

If so, this isn't that hard a fix.  There will need to be a configure test written to determine whether Octave is compiling on a MIPS architecture and then there will need to be an #ifdef in lo-ieee.h to choose between two different definitions of LO_IEEE_NA_HW.

Rik <rik5>
Project Administrator
Sun 10 Jan 2021 04:27:18 PM UTC, comment #12: 

The last test seems to have worked which means this might be a way forward.  According to your post, you tried for verification

Rik <rik5>
Project Administrator
Sun 10 Jan 2021 09:19:49 AM UTC, comment #11: 

comment #8:

> Instead, you might try re-defining the high word of Octave NA so that the quiet signaling bit is '0' to match MIPS.  As another test, I would ask you to try changing liboctave/util/lo-ieee.h.
>

> diff -r e4c152e827aa liboctave/util/lo-ieee.h
> --- a/liboctave/util/lo-ieee.h        Thu Jan 07 18:34:18 2021 +0100
> +++ b/liboctave/util/lo-ieee.h        Fri Jan 08 14:09:31 2021 -0800
> @@ -69,7 +69,7 @@ typedef union
>
>  #define LO_IEEE_NA_HW_OLD 0x7ff00000
>  #define LO_IEEE_NA_LW_OLD 1954
> -#define LO_IEEE_NA_HW 0x7FF840F4
> +#define LO_IEEE_NA_HW 0x7FF040F4
>  #define LO_IEEE_NA_LW 0x40000000
>  #define LO_IEEE_NA_FLOAT   0x7FC207A2
>

>
> Apply the patch with 'patch -p1 < mypatch' and then re-compile.
>
> Try
>

> format native-bit
> x = NA
> y = x / 4
> isna (y)


I recompiled Octave on a Debian mips64el system with Rik's patch above.  It yields this:

octave:1> format native-bit
octave:2> x = NA
x = 0000000000000000000000000000001000101111000000100000111111111110
octave:3> y = x / NA
y = 0000000000000000000000000000001000101111000000100000111111111110
octave:4> isna (y)
ans = 10000000

Just for the record, this is what happens without the patch:

octave:1> format native-bit
octave:2> x = NA
x = 0000000000000000000000000000001000101111000000100001111111111110
octave:3> y = x / NA
y = 1111111111111111111111111111111111111111111111111110111111111110
octave:4> isna (y)
ans = 00000000

Rafael Laboissiere <rlaboiss>
Sat 09 Jan 2021 06:11:41 PM UTC, comment #10: 

The specific value of NA was chosen to be exactly the same as the corresponding value in R so that saving and loading data in binary format would preserve the value when transferring data between Octave and R.  I had initially hoped we would maybe modify functions in Octave to actually respect NA values when appropriate, but that hasn't ever happened since NA was added to Octave nearly 20 years ago.  Maybe it would make more sense to just deprecate and drop NA from Octave?

John W. Eaton <jwe>
Project Administrator
Sat 09 Jan 2021 03:50:39 PM UTC, comment #9: 

On mips64el:

octave:1> format native-bit
octave:2> x = NA
x = 0000000000000000000000000000001000101111000000100001111111111110
octave:3> x / 4
ans = 1111111111111111111111111111111111111111111111111110111111111110

Rafael Laboissiere <rlaboiss>
Fri 08 Jan 2021 10:12:32 PM UTC, comment #8: 

Can you try this?

format native-bit
x = NA
x / 4

Also, one thing I notice is that the silent signaling bit for NaN in IEEE is turned on in Octave according to the IEEE 2008 format.  However, as I noted in comment #6 and is shown in your comment #7, MIPS systems seem to use a value of '0' to indicate quiet signaling by default.

Trying to reverse that convention would require using the '-mnan=2008' option not just for Octave, but for all the libraries it links to.  That sounds hard.

Instead, you might try re-defining the high word of Octave NA so that the quiet signaling bit is '0' to match MIPS.  As another test, I would ask you to try changing liboctave/util/lo-ieee.h.

diff -r e4c152e827aa liboctave/util/lo-ieee.h
--- a/liboctave/util/lo-ieee.h        Thu Jan 07 18:34:18 2021 +0100
+++ b/liboctave/util/lo-ieee.h        Fri Jan 08 14:09:31 2021 -0800
@@ -69,7 +69,7 @@ typedef union

#define LO_IEEE_NA_HW_OLD 0x7ff00000
#define LO_IEEE_NA_LW_OLD 1954
-#define LO_IEEE_NA_HW 0x7FF840F4
+#define LO_IEEE_NA_HW 0x7FF040F4
#define LO_IEEE_NA_LW 0x40000000
#define LO_IEEE_NA_FLOAT   0x7FC207A2

Apply the patch with 'patch -p1 < mypatch' and then re-compile.

Try

format native-bit
x = NA
y = x / 4
isna (y)

Rik <rik5>
Project Administrator
Fri 08 Jan 2021 08:32:01 PM UTC, comment #7: 

Just in case this can be of any interest, here is what I get on a mips64el Debian system:

octave:1> format native-bit
octave:2> NA
ans = 0000000000000000000000000000001000101111000000100001111111111110
octave:3> NaN
ans = 1111111111111111111111111111111111111111111111111110111111111110

and this is what I get on a amd64 Debian system:

octave:1> format native-bit
octave:2> NA
ans = 0000000000000000000000000000001000101111000000100001111111111110
octave:3> NaN
ans = 0000000000000000000000000000000000000000000000000001111111111110

Rafael Laboissiere <rlaboiss>
Fri 08 Jan 2021 07:05:00 PM UTC, comment #6: 

I suspect there isn't going to be a good solution.

See the file liboctave/utils/lo-ieee.h.

Octave defines an NA value to be composed of two 32-bit values.

#define LO_IEEE_NA_HW 0x7FF840F4
#define LO_IEEE_NA_LW 0x40000000

The routine that decides whether something is NA versus NaN is

int
__lo_ieee_is_NA (double x)
{
  lo_ieee_double t;
  t.value = x;
  return (__lo_ieee_isnan (x) && t.word[lo_ieee_hw] == LO_IEEE_NA_HW
          && t.word[lo_ieee_lw] == LO_IEEE_NA_LW) ? 1 : 0;
}

The IEEE standard defines NaN as having the sign bit cleared, maximum possible exponent, and a mantissa which is non-zero.  Thus, the high word begins with 0x7FF.  Most mathematical libraries detect NaN and then don't further process the number.  So when you operate on an NA value in an IEEE-compliant library the value just passes through.  However, this is implementation dependent.  My guess is that the ARM software library, or hardware implementation, detects that the value is a NaN and then substitutes a constant NaN value such as 0x7FF80000.  This value is also NaN according to IEEE standards, but it is no longer an Octave NA because the values have changed.  If you can compile with a debugger you could check whether this is correct.

There may be special switches you can throw during compilation to determine how IEEE operations are implemented (such as not using 80bit precision, etc.).  Maybe there is something that might be useful for MIPS?  In gcc, for example, I see '-mhard-float' to require use of HW math co-processor and '-msoft-float' to use software libraries for floating point.

This option also looks promising

-mnan=2008
       -mnan=legacy
           These options control the encoding of the special not-a-number (NaN) IEEE 754 floating-point
           data.

           The -mnan=legacy option selects the legacy encoding.  In this case quiet NaNs (qNaNs) are
           denoted by the first bit of their trailing significand field being 0, whereas signaling NaNs
           (sNaNs) are denoted by the first bit of their trailing significand field being 1.

           The -mnan=2008 option selects the IEEE 754-2008 encoding.  In this case qNaNs are denoted by
           the first bit of their trailing significand field being 1, whereas sNaNs are denoted by the
           first bit of their trailing significand field being 0.

If you can verify with a debugger first I think that would be the best first step.

Rik <rik5>
Project Administrator
Thu 07 Jan 2021 07:52:13 PM UTC, comment #5: 

comment #4:

> Okay, at least defining an NA value is possible.
>
> Next step is to isolate the problem and see which mathematical operations preserve NA and which convert it to NaN.  Just guessing right now, it is more likely that a hardware accelerated multiplication/division is going to change things.
>
> Tests:
>
> x = NA;
> x + 1
> x - 1
> x * 1
> x / 1


Here we are:

octave:1> x = NA;
octave:2> x + 1
ans = NaN
octave:3> x - 1
ans = NaN
octave:4> x * 1
ans = NaN
octave:5> x / 1
ans = NaN

Rafael Laboissiere <rlaboiss>
Thu 07 Jan 2021 02:43:06 PM UTC, comment #4: 

Okay, at least defining an NA value is possible.

Next step is to isolate the problem and see which mathematical operations preserve NA and which convert it to NaN.  Just guessing right now, it is more likely that a hardware accelerated multiplication/division is going to change things.

Tests:

x = NA;
x + 1
x - 1
x * 1
x / 1

Rik <rik5>
Project Administrator
Thu 07 Jan 2021 11:50:13 AM UTC, comment #3: 

I get this on a mips46el system with Octave 6.1.0 (actually, with Debian package octave version 6.1.1~hg.2020.12.27-2):

octave:1> x = NA
x =  NA
octave:2> isna (x)
ans = 1
octave:3> y = ~x
error: invalid conversion from NaN to logical

Rafael Laboissiere <rlaboiss>
Thu 07 Jan 2021 02:20:05 AM UTC, comment #2: 

Very strange.

On the mips64el system, is every NA translated to NaN?  For example, does this print NA and 1?

x = NA
isna (x)

What happens when you try to perform the logical operation ~x?

y = ~x

I get an error message on Linux.

Rik <rik5>
Project Administrator
Wed 06 Jan 2021 08:28:26 AM UTC, comment #1: 

The BIST in the level-set package that exposes the problem is the one in so_step_armijo.m, as can be seen in the build log of the Debian package octave-level-set on mips64el:

>>>>> /<<PKGBUILDDIR>>/inst/so_step_armijo.m
***** function s = updateState (phi, data)
  inside = ls_inside (phi);
  vol = data.g.h * length (find (inside));
  cost = 1 / vol;

  % Throw if the volume is too large.  This is used
  % to check the error handling code.
  if (vol > 15)
    error ("volume too large");
  endif

  s = struct ("phi", phi, "cost", cost);
***** endfunction
***** shared n, phi, baseData, d, f, dJ
  n = 100;
  x = linspace (-10, 10, n);
  h = x(2) - x(1);
  a = 0;
  b = 5;
  phi = ls_genbasic (x, "box", a, b);

  baseData = struct ();
  baseData.p = so_init_params (false);
  baseData.g = struct ("x", x, "h", h);
  baseData.cb = struct ("update_state", @updateState);
  baseData.s = updateState (phi, baseData);
  baseData.s.phi = phi;

  f = ones (size (phi));
  d = ls_solve_stationary (phi, f, h);

  curVol = b - a;
  dJ = -2 / curVol^2;
!!!!! test failed
assert (all (isna (d0 (:)) | !isnan (d0 (:)))) failed
shared variables   scalar structure containing the fields:

    n = [](0x0)
    phi = [](0x0)
    baseData = [](0x0)
    d = [](0x0)
    f = [](0x0)
    dJ = [](0x0)

These are the lines in ls_solve_stationary.m that cause the bug in mips64el:

  d0 = nb;
  d0(~fz) ./= fabs(~fz);

Rafael Laboissiere <rlaboiss>
Tue 05 Jan 2021 10:10:12 PM UTC, original submission:  

Octave is producing different results for arithmetic operations involving NA in different architectures of Debian GNU/Linux.  On a amd64 system, I get:

$ echo "NA / 1" | octave-cli
ans = NA

while on a mips64el system, I get:

$ echo "NA / 1" | octave-cli
ans = NaN

This is preventing a BIST in the level-set package to succeed on a mips64el system.

Does anyone know the cause for this disparity?

Rafael Laboissiere <rlaboiss>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by jwe (Posted a comment)
  • -email is unavailable- added by rik5 (Posted a comment)
  • -email is unavailable- added by rlaboiss (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can add your encouragement to it.
    This task has 0 encouragements so far.

    Only project members can vote.

     

     

     

    Follows 1 latest change.

    Date Changed by Updated Field Previous Value => Replaced by
    2021-01-07 rik5 StatusNone => Need Info

    Back to the top


    Powered by Savane 3.5