bugGNU Octave - Bugs: bug #65089, assert may become excessively slow...

 
 

bug #65089: assert may become excessively slow in case of large matrices

Submitter:  None
Submitted:  Thu 28 Dec 2023 04:30:36 PM UTC
   
 
Category:  Octave Function Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Performance
Status:  None Assigned to:  None
Originator Name:  octave-user Originator Email:  -email is unavailable-
Open/Closed:  * Open Release:  * 8.4.0
Operating System:  * GNU/Linux Fixed Release:  None
Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Thu 28 Dec 2023 09:26:01 PM UTC, comment #8: 

This would give us the following output:

Elapsed time is 0.025372 seconds.
error: assert failed: 1000000 values are out of tolerance; max difference is 0.997544 > 2.22045e-16; message is too long ...


As you can see, it would be very fast and you can see the most important information without the need to check 1000000 lines of output.

Anonymous
Thu 28 Dec 2023 09:23:09 PM UTC, comment #7: 

What I was thinking about is something like this:

assert_msg_limit=1000;
tol = eps;
N = 1000;
a = rand(N, N);
b = rand(N, N);
tic;
cond_fail=(abs(a - b) > tol);
n_fail=sum(cond_fail(:));
toc;
if (n_fail > assert_msg_limit)
  error("assert failed: %d values are out of tolerance; max difference is %g > %g; message is too long ...", n_fail, max(abs(a - b)(:)), tol);
endif


comment #6:

> Should it just return at the first FALSE?
>
> Dmitri.
> --
>

Anonymous
Thu 28 Dec 2023 09:10:08 PM UTC, comment #6: 

Should it just return at the first FALSE?

Dmitri.
--

Dmitri A. Sergatskov <dasergatskov>
Thu 28 Dec 2023 08:54:08 PM UTC, comment #5: 

This is almost the solution which I mentioned at the beginning, but it would mean that I have to modify a lot of code.
comment #2:

> You might try something like
>


> assert (max (abs (a - b)(:)) < tol)


>

Anonymous
Thu 28 Dec 2023 08:53:07 PM UTC, comment #4: 

Octave's assert function was originally designed to be used in the Octave test suite where most test expressions have reasonable sizes.

But yes, it could be useful to limit the output that is displayed for this calling form.

John W. Eaton <jwe>
Group administrator
Thu 28 Dec 2023 08:50:15 PM UTC, comment #3: 

Well, I think the main problem is, that it takes so much time and memory to build the error message. Actually, it may take hours without getting any output on the screen. But of course, this might be just a matter of the matrix size N.

Anonymous
Thu 28 Dec 2023 08:49:57 PM UTC, comment #2: 

You might try something like


assert (max (abs (a - b)(:)) < tol)


John W. Eaton <jwe>
Group administrator
Thu 28 Dec 2023 08:29:18 PM UTC, comment #1: 

The slowness is because of the display of two million values. Octave is only doing what you asked with assert.


Anonymous
Thu 28 Dec 2023 04:30:36 PM UTC, original submission:  

Dear GNU-Octave developers,

If you execute the following code in Octave, it will take forever.


tol = eps;
N = 1000;
a = rand(N, N);
b = rand(N, N);
assert(a, b, tol);


Obviously, one could rewrite the above code as follows:

tol = eps;
N = 1000;
a = rand(N, N);
b = rand(N, N);
assert(max(max(abs(a-b)))<=tol);


However, if you have a large testsuite with many calls to assert, your whole testsuite might be broken, just because a single assert fails. Rewriting the whole testsuite might be tedious. Now I'm considering the option to replace all my calls to assert by a custom version of assert which does not have such a performance issue.

Nevertheless, I think that it would be a good idea to limit somehow the length of the error message generated by assert, because nobody would read an error message with one million lines. For example, maximum 1000 differences larger than the tolerance could be reported. As an alternative, assert could just report the maximum error or the total number of values with a difference above the tolerance. A remark could be added, that the complete error message would be too long to display it.

Please let me know if such an option would be acceptable for you!
In that case I could create and submit a patch for assert.

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by dasergatskov (Posted a comment)
  • -email is unavailable- added by jwe (Posted a comment)
  • -email is unavailable- added by None (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    No changes have been made to this item

    Back to the top

    Powered by Savane 3.13-caa5.
    Corresponding source code