bugGNU Octave - Bugs: bug #54956, [octave forge] (interval) Unit...

 
 

bug #54956: [octave forge] (interval) Unit test in mpfr_matrix_mul_d.cc fails non-deterministically

Submitter:  Rafael Laboissière <rlaboiss>
Submitted:  Sun 04 Nov 2018 10:33:30 PM UTC
   
 
Category:  Octave Package Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Other
Status:  Fixed Assigned to:  None
Originator Name:  Open/Closed:  * Closed
Release:  * 4.4.1 Operating System:  * Any
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Mon 07 Feb 2022 12:23:14 AM UTC, comment #9: 

Thanks for the fix Oliver 🙂👍

Kai Torben Ohlhus <siko1056>
Group Member
Sun 06 Feb 2022 10:55:58 PM UTC, comment #8: 
Oliver Heimlich <oheim>
Mon 07 Jan 2019 10:17:03 AM UTC, comment #7: 

Regarding comment #5 and comment #6, yes I also vote to remove this test.

Kai Torben Ohlhus <siko1056>
Group Member
Sun 06 Jan 2019 11:01:17 PM UTC, comment #6: 

Kai - as a general response to your comments about not being able to reproduce, just remember that Octave is not running on a a RTOS, and imagine that Octave gets suspended from running for a few seconds at some random time during the execution of this test.

Mike Miller <mtmiller>
Group Member
Sun 06 Jan 2019 10:58:37 PM UTC, comment #5: 

I think the test should be removed entirely. Besides being non-deterministic and being possible to fail completely at random, this test also relies on the private internal function '__octave_config_info__' which should not be used by any code outside of Octave itself.

It could be turned into a demo, with a simple explanatory comment that the user will see explaining that it depends on OpenMP and that it may fail if the system load is not uniform while the demo is running.

Mike Miller <mtmiller>
Group Member
Fri 23 Nov 2018 07:34:19 AM UTC, comment #4: 

A simple fix is to transform the unit test from "%!test" into  "%!xtest", as I suggested in my patch.

Rafael Laboissière <rlaboiss>
Thu 22 Nov 2018 05:32:55 PM UTC, comment #3: 

Sure, I ran your original example for 10 minutes and never missed the factor 1.2 condition [1] intended by Oliver ^^  I was just curious about the timings of your system.  Maybe there was some peculiar setting or behavior that Oliver can consider when working on this item...

So I think Oliver it is your turn to avoid timing issues ;-)

[1] https://sourceforge.net/p/octave/interval/ci/default/tree/src/mpfr_matrix_mul_d.cc#l178

Kai Torben Ohlhus <siko1056>
Group Member
Thu 22 Nov 2018 04:50:36 PM UTC, comment #2: 

I have Octave 4.4.1 and interval 3.2.0.  This is what I get for your example:


> nproc ("overridable")
ans =  4
> tic; mpfr_matrix_mul_d (A', A, A', A); time_per_element = toc
time_per_element =  3.4102
> tic; mpfr_matrix_mul_d (A', A, A', A); time_per_element_parallel = toc / numel (ans)
time_per_element_parallel =  0.85000


However, when I launch again the same example, I get:


> tic; mpfr_matrix_mul_d (A', A, A', A); time_per_element = toc
time_per_element =  1.3613
> tic; mpfr_matrix_mul_d (A', A, A', A); time_per_element_parallel = toc / numel (ans)
time_per_element_parallel =  0.97579


Do you see?  The problem here is that this unit test is not deterministic.

Rafael Laboissière <rlaboiss>
Thu 22 Nov 2018 01:46:39 PM UTC, comment #1: 

Added Oliver as package maintainer.

Personally, I cannot reproduce your problem on Linux with Octave 4.4.1 and interval 3.2.0.  Which Octave version do you use and what is the output for you running the test?


__octave_config_info__.ENABLE_OPENMP

nproc ("overridable")

A = vec (magic (2000));

tic; mpfr_matrix_mul_d (A', A, A', A); time_per_element = toc

A = [A A];

tic; mpfr_matrix_mul_d (A', A, A', A); time_per_element_parallel = toc / numel (ans)


My system with Octave 4.4.1 "says":


>> __octave_config_info__.ENABLE_OPENMP
ans = 1
>> nproc ("overridable")
ans =  4
time_per_element =  3.4767
time_per_element_parallel =  0.65099


So at least my system "behaves" as Oliver expected with this test...

Kai Torben Ohlhus <siko1056>
Group Member
Sun 04 Nov 2018 10:33:30 PM UTC, original submission:  

The second unit test in file src/mpfr_matrix_mul_d.cc of the octave-interval package fails in a non-deterministic way.  The problem can be triggered like this:


$ echo 'pkg load interval ; while 1 ; test("src/mpfr_matrix_mul_d.cc") ; endwhile' | octave-cli -q 2>&1 | grep failed
!!!!! test failed
assert (time_per_element > 1.2 * time_per_element_parallel) failed


The patch attached to this bug report fixes the problem.

Rafael Laboissière <rlaboiss>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files

 

Depends on the following items: None found

Digest:
   bug dependencies.

 

Carbon-Copy List
  • -email is unavailable- added by oheim (Posted a comment)
  • -email is unavailable- added by siko1056 (Posted a comment)
  • -email is unavailable- added by siko1056 (Package maintainer)
  • -email is unavailable- added by rlaboiss (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 9 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2022-02-07 siko1056 StatusNeed Info Fixed
        Open/ClosedOpen Closed
    2019-01-06 mtmiller Dependencies- bugs #55407 is dependent
    2018-11-22 siko1056 Releaseother 4.4.1
    2018-11-22 siko1056 StatusNone Need Info
    2018-11-22 siko1056 Carbon-Copy- Added oheim
    2018-11-22 siko1056 Carbon-CopyRemoved oliver heimlich <oheim@posteo.de> -
    2018-11-22 siko1056 Carbon-Copy- Added oliver heimlich <oheim@posteo.de>
    2018-11-04 rlaboiss Attached File- Added xtest-in-mpfr-matrix-mul-d.patch, #45360

    Back to the top

    Powered by Savane 3.13-758e.
    Corresponding source code