bugGNU Octave - Bugs: bug #55250, test suite severely overcounts...

 
 

bug #55250: test suite severely overcounts "files with no tests"

Submitter:  Mike Miller <mtmiller>
Submitted:  Thu 20 Dec 2018 12:08:46 AM UTC
   
 
Category:  Test Suite Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Incorrect Result
Status:  Fixed Assigned to:  None
Originator Name:  Open/Closed:  * Closed
Release:  * dev Operating System:  * GNU/Linux
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Fri 21 Dec 2018 06:27:38 PM UTC, comment #5: 

Rebased on the stable branch and pushed

https://hg.savannah.gnu.org/hgweb/octave/rev/01fdb337fa20

Mike Miller <mtmiller>
Group Member
Thu 20 Dec 2018 10:08:59 PM UTC, comment #4: 

Yes, it's fine to go ahead with this change.

Thanks.

John W. Eaton <jwe>
Group administrator
Thu 20 Dec 2018 10:06:08 PM UTC, comment #3: 

Thanks, your patch fixes the problem for me as well:


163 (of 837) .m files have no tests.


Please push the patch.

Kai Torben Ohlhus <siko1056>
Group Member
Thu 20 Dec 2018 07:34:26 PM UTC, comment #2: 

The attached change fixes this problem for me. It's basically introducing a boolean flag to choose between files that end in .tst or files that end in .m and -tst, depending on which group of directories the function is called on. I'll push in a couple hours if I don't hear any negative feedback.

(file #45706)

Mike Miller <mtmiller>
Group Member
Thu 20 Dec 2018 07:09:55 PM UTC, comment #1: 

In changeset 2be7cf62b9ee (https://hg.savannah.gnu.org/hgweb/octave/rev/2be7cf62b9ee), the '__run_test_suite__' function was refactored to eliminate some code duplication. Unfortunately this also deleted the distinction between "integrated test scripts" and "fixed test scripts". The previous revision only considered files ending in the .tst file extension under the fixed test directory(ies).

I would like to restore that distinction, so that only certain file types are checked depending on whether it is called on 'fcndirs' or 'fixedtestdirs'. Maybe a third parameter to the run_test_dir subfunction?

Mike Miller <mtmiller>
Group Member
Thu 20 Dec 2018 12:08:46 AM UTC, original submission:  

On the default branch, the test suite overcounts the number of .m files that have no built-in tests. This is because it seems to be looking at .m files in the 'test' directory now, where these were not considered before. This is a regression from the stable branch.

For example, on a recent 'make check' on the stable branch (or Octave version 4.4.1), the end of the output reports


164 (of 821) .m files have no tests.


On the current default branch (01f1e70c80b6), 'make check' reports


367 (of 1041) .m files have no tests.


The number of .m files under the top-level 'test' directory is 192. 4 of these are under a "private" directory, so that leaves 188. And these files do not show up in the tail section of fntests.log on the stable branch, but all 188 files do show up in fntests.log on the default branch.

Mike Miller <mtmiller>
Group Member

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #45706:  test.diff added by mtmiller (2KiB - text/x-patch)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by jwe (Posted a comment)
  • -email is unavailable- added by siko1056 (Posted a comment)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 7 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2018-12-21 mtmiller StatusPatch Reviewed Fixed
        Open/ClosedOpen Closed
    2018-12-20 siko1056 StatusPatch Submitted Patch Reviewed
    2018-12-20 mtmiller Attached File- Added test.diff, #45706
        StatusIn Progress Patch Submitted
    2018-12-20 mtmiller StatusNone In Progress
    2018-12-20 mtmiller Carbon-CopyRemoved 80942 -

    Back to the top

    Powered by Savane 3.13-02a9.
    Corresponding source code