# bug #57175: Incorrect output using rref() function on integer matrix

 Submitted by: None Submitted on: Wed 06 Nov 2019 04:33:10 AM UTC Category: Octave Function Severity: 3 - Normal Priority: 3 - Low Item Group: Incorrect Result Status: Invalid Assigned to: None Originator Name: Josh Originator Email: -email is unavailable- Open/Closed: Closed Release: 5.1.0 Operating System: Any

You are not logged in

## Discussion

Thu 07 Nov 2019 02:41:07 AM UTC, comment #3:

If the A matrix contains 628 instead of 622 in the last element, then the integer solution [-3; -5; 2; 4] works. I'm going to guess that this was a typo in the original example and close this as not a bug.

Mike Miller <mtmiller>
Wed 06 Nov 2019 05:29:03 PM UTC, comment #2:

I agree that some kind of integer rounding must be in effect with your claim of what the correct answer ought to be.

For one more example, I used the symbolic package to compute the answer as

R = (sym 4×5 matrix)

⎡1  0  0  0  -13/4⎤
⎢                 ⎥
⎢            -99  ⎥
⎢0  1  0  0  ──── ⎥
⎢             20  ⎥
⎢                 ⎥
⎢0  0  1  0    2  ⎥
⎢                 ⎥
⎣0  0  0  1    4  ⎦

Mike Miller <mtmiller>
Wed 06 Nov 2019 07:22:45 AM UTC, comment #1:

Thank you for the bug report.  Same can be observed with the current stable Octave 5.1.0 release.

Please note, that in your example "A" is a double matrix, as it is the default data type in Octave.  Use for example

A = int32 ([0,0,1,1,6;1,5,0,0,-28;0,0,-1,-25,-102;-1,-125,0,0,622])

to get a true integer matrix.

Unfortunately for your case, rref() is only defined for "double" and "single" input.  This is true for Matlab R2019a as well, and the result there is the same as in Octave.

To get a matrix rounded to integer like double values, please use

A = round (rref (A))

Does this satisfy your need?  I did not explicitly calculate your example by hand.  But stepping through the function rref() with the debugger, I cannot spot any wrong intermediate steps.

To resolve this bug report please clarify the following two questions:

- Did you only expect a "true integer" output for your "integer stored inside a double" input?

Kai Torben Ohlhus <siko1056>
Wed 06 Nov 2019 04:33:10 AM UTC, original submission:

I attempted running the following lines:

A = [0,0,1,1,6;1,5,0,0,-28;0,0,-1,-25,-102;-1,-125,0,0,622]
rref(A)

The correct answer ought to be:
1.00000   0.00000   0.00000   0.00000  -3.00000
0.00000   1.00000  -0.00000  -0.00000  -5.00000
0.00000   0.00000   1.00000   0.00000   2.00000
0.00000   0.00000   0.00000   1.00000   4.00000

But is returned as:
1.00000   0.00000   0.00000   0.00000  -3.25000
0.00000   1.00000  -0.00000  -0.00000  -4.95000
0.00000   0.00000   1.00000   0.00000   2.00000
0.00000   0.00000   0.00000   1.00000   4.00000

Anonymous

## Attached Files

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:

Comment:

No files currently attached

## Dependencies

Depends on the following items: None found

Items that depend on this one: None found

Carbon-Copy List
• -email is unavailable- added by siko1056 (Posted a comment)
•

Do you think this task is very important?
This task has 0 encouragements so far.

Only project members can vote.