============================================================= PEG bug_test--tjl: Handling bugs ============================================================= :Author: Tuomas Lukka :Last-Modified: $Date: 2002/11/17 11:04:52 $ :Revision: $Revision: 1.1 $ :Status: Incomplete Making failing tests for bugs is not currently handled very well. This PEG proposes a relatively simple way to handle it. Issues ====== * Should we use the test name or the docstring for tagging? * What conventions should there be for encoding the failing of tests? * Should we be able to tag separately tests failing in AWT, tests failing in GL and both? RESOLVED: Yes. It's very good to be able to say in the test that "this works in GL, but doesn't in AWT" so that the normal testrunner *will* run the test for GL. Bug tests ========= Starting from the following: 1) Unit tests are important for seeing if things still work correctly, after a given change. Therefore, it must be clear whether things went in the expected way or not. 2) We want to be able to make failing tests for bugs before fixing them 3) Making failing tests in separate files or so is impractical and can lead to mistakes when moving tests (e.g. the framework). The proposal is that we encode whether a given test is failing into the test name. For example, :: def testFoo(): # run test always pass def testBar__failAWT(): # this test is known to fail on AWT. pass Then, the Makefile targets should be adjusted as follows:: make test # Run all tests that should succeed on AWT make test-gl # Run all tests that should succeed on GL make testbugs # Run all tests that should fail on AWT make testbugs-gl # Run all tests that should fail on GL