bugGNU Octave - Bugs: bug #63229, Missing self test for option...


bug #63229: Missing self test for option OutputSel in ode23, ode23s, ode45

Submitter:  Ken Marek <m5318>
Submitted:  Mon 17 Oct 2022 07:02:02 PM UTC
Category:  Octave Function Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Other
Status:  Fixed Assigned to:  None
Originator Name:  Ken Marek Open/Closed:  * Closed
Release:  * dev Operating System:  * Any
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup

Jump to the original submission

Mon 21 Nov 2022 04:02:49 PM UTC, comment #10: 

I pushed the changeset to stable and merged with default:


Marking as ready for test.

John W. Eaton <jwe>
Group administrator
Thu 17 Nov 2022 11:56:40 PM UTC, comment #9: 

Honestly, with the tests being split out individually, I think it's actually okay the way it is. If any test throws an error it does show pretty clearly which test caused the error and which assert statement inside the function was tripped.

I'm not sure exactly what happens inside the test environment, but I don't think creating globals are necessarily the best way to go. This function uses a persistent variable, I don't know how that does or doesn't function in the test environment compared to in a normal function either, but it didn't seem to cause problems.

Nicholas Jankowski <nrjank>
Group Member
Thu 17 Nov 2022 05:22:38 PM UTC, comment #8: 

One more alternative has occurred to me. We could set a global variable in the OutputSel_test function, then check its value(s) and do the asserts in the main test block. Again, I'm not sure that would be considered less messy than what we've got now, but it is different.

Ken Marek <m5318>
Thu 17 Nov 2022 04:13:25 AM UTC, comment #7: 

ok.  I played around with it a bit, I also didn't see a better way to test that functionality than how you did it. all 6 asserts get hit in the tests you created, so I don't think it's any messier than the rest of the tests in those functions.

the only changes I made were to divide up the test block into 3 individual test blocks. this will make it a bit clearer which block is creating the error if one occurs.

adjusted patch attached. passes all tests, and I remembered to keep your name on it :)

Requesting someone else push it if it looks good, though. It's just a BIST change and should be fairly low impact, but I'm not sure where things are right now in the current state of octave 8 release after the branches were merged.

(file #53967)

Nicholas Jankowski <nrjank>
Group Member
Wed 16 Nov 2022 11:57:38 PM UTC, comment #6: 

No, we shouldn't change the functionality of the source just to make testing easier without really good reason. I'll try to take a look at it soon.

Nicholas Jankowski <nrjank>
Group Member
Wed 16 Nov 2022 10:26:03 PM UTC, comment #5: 

I agree that this is generally not the preferred form for the test, but I'm not sure any alternatives are more appealing. The problem is that the output of OutputSel function is not returned directly to the outputs of ode{xx}.

Other than just not having a test (the status quo), the only other way to test the OutputSel implementation without changing the code is to have the OutputFcn return the stop signal (stop_solve) if there is a problem. We could then test whether the final output of ode{xx} is the expected value, and infer an error if not. If there is an error, we know that there was a failure in the intermediate steps (flag==""). However, the return value of stop_solve is ignored in the flag=="init" and flag=="done" calls, so without raising an error in OutputFcn itself, problems in those calls are not testable.

I suppose another alternative would be to change the source code to query the value of stop_solve more more than just true/false at each OutputFcn call, but I'm pretty sure that would break Matlab compatibility and generally cause more problems than it fixes. Or we could pass any nonzero return values of stop_solve down the line and put it into the output struct of ode{xx}. It would be extra code/overhead and an undocumented feature, but probably wouldn't break compatibility.

Ken Marek <m5318>
Wed 16 Nov 2022 09:52:43 PM UTC, comment #4: 

yes, I was looking at it recently. I understand the idea that you're adding a common function to get used in multiple tests.  I'm not certain that having the function itself call the assert statements is the right way to go.  I've not seen it done that way before. I was going to try to force some test errors to see how it behaved when calling e.g., 'test ode23'. Generally, I would think it best for the function to return a value that the calling test line will check with an assert statement. 

But maybe someone else has an opinion on whether either form is preferred or problematic?

Nicholas Jankowski <nrjank>
Group Member
Wed 16 Nov 2022 09:44:07 PM UTC, comment #3: 

Now that the dependency bug is marked fixed, presumably this one is ready for testing.

Ken Marek <m5318>
Mon 17 Oct 2022 07:48:22 PM UTC, comment #2: 

This is independent of bug #63162. I do agree we probably need the dependency on bug #63063 patches, although I haven't actually tried to see if this fails without it.

Ken Marek <m5318>
Mon 17 Oct 2022 07:43:07 PM UTC, comment #1: 

updating status and adding the dependency on bug #63063.  Does this also need to be applied after any patches in bug #63162 or are they independent?

Nicholas Jankowski <nrjank>
Group Member
Mon 17 Oct 2022 07:02:02 PM UTC, original submission:  

There is currently a placeholder for the OutputSel self test in functions ode23, ode23s, and ode45. Attached is a patch that tests that option's functionality.

Ken Marek <m5318>


(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:

Attached Files
file #53967:  OutputSeltest_bug63229_v2.patch added by nrjank (7KiB - application/octet-stream - slightly updated patch with split test blocks)
file #53870:  outputsel_20221017.patch added by m5318 (6KiB - application/octet-stream)


   bug dependencies.

Items that depend on this one: None found


Carbon-Copy List
  • -email is unavailable- added by rik5 (Updated the item)
  • -email is unavailable- added by jwe (Posted a comment)
  • -email is unavailable- added by nrjank (Posted a comment)
  • -email is unavailable- added by m5318 (Submitted the item)

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.


    Follow 10 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2023-02-11 rik5 StatusReady For Test Fixed
        Open/ClosedOpen Closed
    2022-11-21 jwe StatusPatch Reviewed Ready For Test
    2022-11-17 nrjank StatusPatch Submitted Patch Reviewed
    2022-11-17 nrjank Attached File- Added OutputSeltest_bug63229_v2.patch, #53967
        Originator Name Ken Marek
    2022-10-17 nrjank StatusNone Patch Submitted
        Release7.2.0 dev
        Dependencies- Depends on bugs #63063
    2022-10-17 m5318 Attached File- Added outputsel_20221017.patch, #53870

    Back to the top

    Powered by Savane 3.13-04b1.
    Corresponding source code