Our first automated tests

Running the test for the calc example

Now we are ready to call the automated tests

Example 7. Sample output of runtest in a configured directory

dgt:~/dejagnu.test$ make check 
make check-DEJAGNU 
make[1]: Entering directory `/home/dgt/dejagnu.test' srcdir=`cd . && pwd`; export srcdir; \ 
EXPECT=expect; export EXPECT; \ runtest=runtest; \ 
if /bin/sh -c "$runtest --version" > /dev/null 2>&1; then \
$runtest --tool calc CALC=`pwd`/calc --srcdir ./testsuite ; \ 
else echo "WARNING: could not find \`runtest'" 1>&2; :;\ 
fi 
WARNING: Couldn't find the global config file. 
WARNING: Couldn't find tool init file 
Test Run By dgt on Sun Nov 25 21:42:21 2001 
Native configuration is i586-pc-linux-gnu

       === calc tests ===

Schedule of variations: 
   unix

Running target unix 
Using /usr/share/dejagnu/baseboards/unix.exp as board description file for target. 
Using /usr/share/dejagnu/config/unix.exp as generic interface file for target. 
Using ./testsuite/config/unix.exp as tool-and-target-specific interface file.
Running ./testsuite/calc.test/calc.exp ... 
FAIL: multiply2 (bad match)

=== calc Summary ===

# of expected passes 5 
# of unexpected failures 1 
/home/Dgt/dejagnu.test/calc version Version: 1.1
make[1]: *** [check-DEJAGNU] Fehler 1 
make[1]: Leaving directory `/home/Dgt/dejagnu.test' make: *** [check-am] Fehler 2

Did you see the line “FAIL:“? The test cases for calc catch the bug in the calc.c file. Fix the error in calc.c later as the following examples assume a unchanged calc.c.

Examine the output files calc.sum and calc.log. Try to understand the testcases written in ~/dejagnu.test/testsuite/calc.test/calc.exp. To understand Expect you might take a look at the book "Exploring Expect", which is an excellent resource for learning and using Expect. (Pub: O'Reilly, ISBN 1-56592-090-2) The book contains hundreds of examples and also includes a tutorial on Tcl. Exploring Expect is 602 pages long. 

The various config files or how to avoid warnings

DejaGnu may be customized by each user. It first searches for a file called ~/.dejagnurc. Create the file ~/.dejagnurc and insert the following line:

puts "I am ~/.dejagnurc"

Rerun make check. Test whether the output contains "I am ~/.dejagnurc". Create ~/my_dejagnu.exp and insert the following line:

puts "I am ~/my_dejagnu.exp"

In a Bash-Shell enter

dgt:~/dejagnu.test$ export DEJAGNU=~/my_dejagnu.exp

Run “make check” again. The output should not contain “WARNING: Couldn't find the global config file.”. Create the sub-directory lib. Create the file “calc.exp” in it and insert the following line:

puts "I am lib/calc.exp"

The last warning “WARNING: Couldn't find tool init file” should not be part of the output of make check. Create the directory ˜/boards. Create the file ˜/boards/standard.exp and insert the following line:

puts "I am boards/standard.exp"

If the variable DEJAGNU is still not empty then the (abbreviated) output of “make check” should look like this:

Example 8. Sample output of runtest with the usual configuration files

dgt:~/dejagnu.test$ make check 
<...>
fi 
I am ~/.dejagnurc 
I am ~/my_dejagnu.exp 
I am lib/calc.exp 
Test Run By dgt on Sun Nov 25 22:19:14 2001 
Native configuration is i586-pc-linux-gnu

     === calc tests ===
Using /home/Dgt/boards/standard.exp as standard board description\
file for build. 
I am ~/boards/standard.exp 
Using /home/Dgt/boards/standard.exp as standard board description\
 file for host. 
I am ~/boards/standard.exp 

Schedule of variations: 
  unix

Running target unix 
Using /home/Dgt/boards/standard.exp as standard board description\
 file for target. 
I am ~/boards/standard.exp 
Using /usr/share/dejagnu/baseboards/unix.exp as board description file\
for target. 
<...>

It is up to you to decide when and where to use any of the above mentioned config files for customizing. This chapters showed you where and in which order the different config files are run.

When trouble strikes

Calling runtest with the '-v'-flag shows you in even more details which files are searched in which order. Passing it several times gives more and more details.

Example 9. Displaying details about runtest execution

runtest -v -v -v --tool calc CALC=`pwd`/calc --srcdir ./testsuite

Calling runtest with the '--debug'-flag logs a lot of details to dbg.log where you can analyse it afterwards.

In all test cases you can temporary adjust the verbosity of information by adding the following Tcl-command:

set verbose 9

Testing “Hello world” locally

This test checks, whether the built-in shell command “echo Hello world” will really write “Hello world” on the console. Create the file ~/dejagnu.test/testsuite/calc.test/local_echo.exp. It should contain the following lines

Example 10. A first (local) test case

set test "Local Hello World" 
send "echo Hello World" 
expect { 
   -re "Hello World"  { pass "$test" } 
}

Run runtest again and verify the output “calc.log”