1 |
<chapter id=gettingup> |
<chapter id=gettingup> |
2 |
<title>Getting DejaGnu up and running</title> |
<title>Getting DejaGnu up and running</title> |
3 |
<para>This chapter was originally written by Niklaus Giger (ngiger@mus.ch) because he lost a week to figure out how DejaGnu works and how to write a first test.</para> |
<para>This chapter was originally written by Niklaus Giger (ngiger@mus.ch) because he lost a week to figure out how DejaGnu works and how to write a first test.</para> |
4 |
|
|
5 |
<para>Follow these instructions as closely a possible in order get a good insight into how DejaGnu works, else you might run into a lot of subtle problems. You have been warned.</para> |
<para>Follow these instructions as closely a possible in order get a good insight into how DejaGnu works, else you might run into a lot of subtle problems. You have been warned.</para> |
6 |
<para>It should be no big problems installing DejaGnu using your package manager or from the source code. Under a Debian/GNU/Linux systems just type (as root) <programlisting>apt-get dejagnu</programlisting>. These examples were run on a primary machine with a AMD K6 and a Mac Powerbook G3 serving as a remote target.</para> |
<para>It should be no big problems installing DejaGnu using your package manager or from the source code. Under a Debian/GNU/Linux systems just type (as root) <programlisting>apt-get dejagnu</programlisting>. These examples were run on a primary machine with a AMD K6 and a Mac Powerbook G3 serving as a remote target.</para> |
7 |
|
|
8 |
<para> The tests for Windows were run under Windows NT using the actual Cygwin version (1.3.x as of October 2001). It's target system was a PPC embedded system running vxWorks. |
<para> The tests for Windows were run under Windows NT using the actual Cygwin version (1.3.x as of October 2001). It's target system was a PPC embedded system running vxWorks. |
9 |
</para> |
</para> |
10 |
|
|
11 |
<sect1> |
<sect1> |
25 |
</title> |
</title> |
26 |
|
|
27 |
<programlisting> |
<programlisting> |
28 |
dgt:~/dejagnu.test$ runtest |
dgt:~/dejagnu.test$ runtest |
29 |
WARNING: Couldn't find the global config file. |
WARNING: Couldn't find the global config file. |
30 |
WARNING: No tool specified Test |
WARNING: No tool specified Test |
31 |
Run By dgt on Sun Nov 25 17:07:03 2001 Native configuration is i586-pc-linux-gnu |
Run By dgt on Sun Nov 25 17:07:03 2001 Native configuration is i586-pc-linux-gnu |
32 |
=== tests === |
=== tests === |
33 |
Schedule of variations: unix |
Schedule of variations: unix |
34 |
Running target unix Using /usr/share/dejagnu/baseboards/unix.exp as board description file for target. |
Running target unix Using /usr/share/dejagnu/baseboards/unix.exp as board description file for target. |
35 |
Using /usr/share/dejagnu/config/unix.exp as generic interface file for target. |
Using /usr/share/dejagnu/config/unix.exp as generic interface file for target. |
36 |
ERROR: Couldn't find tool config file for unix. |
ERROR: Couldn't find tool config file for unix. |
37 |
=== Summary ===</programlisting> |
=== Summary ===</programlisting> |
38 |
|
|
39 |
<para>We will show you later how to get rid of all the WARNING- and ERROR-messages. The files testrun.sum and testrun.log have been created, which do not interest us at this point. Let's remove them.</para> |
<para>We will show you later how to get rid of all the WARNING- and ERROR-messages. The files testrun.sum and testrun.log have been created, which do not interest us at this point. Let's remove them.</para> |
40 |
|
|
41 |
<programlisting>:~/dejagnu.test$ rm testrun.sum testrun.log |
<programlisting>:~/dejagnu.test$ rm testrun.sum testrun.log |
42 |
</programlisting> |
</programlisting> |
43 |
</example> |
</example> |
44 |
|
|
53 |
<sect2> |
<sect2> |
54 |
<title>Getting the source code for the calc example</title> |
<title>Getting the source code for the calc example</title> |
55 |
|
|
56 |
<para>If you are running a Debian distribution you can find the examples under /usr/share/doc/dejagnu/examples. |
<para>If you are running a Debian distribution you can find the examples under /usr/share/doc/dejagnu/examples. |
57 |
These examples seem to be missing in Red Hat's RPM. |
These examples seem to be missing in Red Hat's RPM. |
58 |
In this case download the sources of DejaGnu and adjust the pathes to the DejaGnu examples accordingly.</para> |
In this case download the sources of DejaGnu and adjust the pathes to the DejaGnu examples accordingly.</para> |
59 |
</sect2> |
</sect2> |
60 |
</sect1> |
</sect1> |
62 |
<sect1> |
<sect1> |
63 |
<title>Create a minimal project, e.g. calc</title> |
<title>Create a minimal project, e.g. calc</title> |
64 |
|
|
65 |
<para>In this section you will to start a small project, |
<para>In this section you will to start a small project, |
66 |
using the sample application calc, which is part of your DejaGnu distribution</para> |
using the sample application calc, which is part of your DejaGnu distribution</para> |
67 |
|
|
68 |
<sect2><title>A simple project without the GNU autotools</title> |
<sect2><title>A simple project without the GNU autotools</title> |
71 |
|
|
72 |
<para>The generated site.exp should like like:</para> |
<para>The generated site.exp should like like:</para> |
73 |
<programlisting> |
<programlisting> |
74 |
set tool calc |
set tool calc |
75 |
set srcdir . |
set srcdir . |
76 |
set objdir /home/dgt/dejagnu.test |
set objdir /home/dgt/dejagnu.test |
77 |
</programlisting></sect2> |
</programlisting></sect2> |
78 |
|
|
96 |
dgt:~/dejagnu.test$ aclocal |
dgt:~/dejagnu.test$ aclocal |
97 |
</programlisting> |
</programlisting> |
98 |
|
|
99 |
<para>autoconf is another part of the auto-tools. |
<para>autoconf is another part of the auto-tools. |
100 |
Run it to generate configure based on information contained in configure.in.</para> |
Run it to generate configure based on information contained in configure.in.</para> |
101 |
|
|
102 |
<programlisting> |
<programlisting> |
103 |
dgt:~/dejagnu.test$ autoconf |
dgt:~/dejagnu.test$ autoconf |
104 |
</programlisting> |
</programlisting> |
105 |
|
|
106 |
<para>autoheader is another part of the auto-tools. |
<para>autoheader is another part of the auto-tools. |
107 |
Run it to generate calc.h.in. </para> |
Run it to generate calc.h.in. </para> |
108 |
|
|
109 |
<programlisting> |
<programlisting> |
110 |
dgt:~/dejagnu.test$ autoheader |
dgt:~/dejagnu.test$ autoheader |
111 |
</programlisting> |
</programlisting> |
112 |
|
|
113 |
<para>The Makefile.am of this example was developed as port of the DejaGnu |
<para>The Makefile.am of this example was developed as port of the DejaGnu |
114 |
distribution. |
distribution. |
115 |
Adapt Makefile.am for this test. Replace the line |
Adapt Makefile.am for this test. Replace the line |
116 |
“#noinst_PROGRAMS = calc” to |
“#noinst_PROGRAMS = calc” to |
117 |
“bin_PROGRAMS = calc”. |
“bin_PROGRAMS = calc”. |
118 |
Change the RUNTESTDEFAULTFLAGS from |
Change the RUNTESTDEFAULTFLAGS from |
119 |
“$$srcdir/testsuite” to |
“$$srcdir/testsuite” to |
120 |
“./testsuite”.</para> |
“./testsuite”.</para> |
121 |
|
|
122 |
<para>Running automake at this point contains a series of warning in its output as shown in the following example:</para> |
<para>Running automake at this point contains a series of warning in its output as shown in the following example:</para> |
125 |
<title>Sample output of automake with missing files</title> |
<title>Sample output of automake with missing files</title> |
126 |
<programlisting> |
<programlisting> |
127 |
dgt:~/dejagnu.test$ automake --add-missing |
dgt:~/dejagnu.test$ automake --add-missing |
128 |
automake: configure.in: installing `./install-sh' |
automake: configure.in: installing `./install-sh' |
129 |
automake: configure.in: installing `./mkinstalldirs' |
automake: configure.in: installing `./mkinstalldirs' |
130 |
automake: configure.in: installing `./missing' |
automake: configure.in: installing `./missing' |
131 |
automake: Makefile.am: installing `./INSTALL' |
automake: Makefile.am: installing `./INSTALL' |
132 |
automake: Makefile.am: required file `./NEWS' not found |
automake: Makefile.am: required file `./NEWS' not found |
133 |
automake: Makefile.am: required file `./README' not found |
automake: Makefile.am: required file `./README' not found |
134 |
automake: Makefile.am: installing `./COPYING' |
automake: Makefile.am: installing `./COPYING' |
135 |
automake: Makefile.am: required file `./AUTHORS' not found |
automake: Makefile.am: required file `./AUTHORS' not found |
136 |
automake: Makefile.am: required file `./ChangeLog' not found |
automake: Makefile.am: required file `./ChangeLog' not found |
137 |
configure.in: 4: required file `./calc.h.in' not found |
configure.in: 4: required file `./calc.h.in' not found |
138 |
Makefile.am:6: required directory ./doc does not exist |
Makefile.am:6: required directory ./doc does not exist |
139 |
</programlisting> |
</programlisting> |
140 |
</example> |
</example> |
141 |
|
|
142 |
<para>Create a empty directory doc and empty files |
<para>Create a empty directory doc and empty files |
143 |
INSTALL, NEWS, README, AUTHORS, ChangeLog and COPYING. |
INSTALL, NEWS, README, AUTHORS, ChangeLog and COPYING. |
144 |
The default COPYING will point to the GNU Public License (GPL). |
The default COPYING will point to the GNU Public License (GPL). |
145 |
In a real project it would be time to add some meaningfull text in each file. |
In a real project it would be time to add some meaningfull text in each file. |
146 |
</para> |
</para> |
150 |
<title>Sample output of configure |
<title>Sample output of configure |
151 |
</title> |
</title> |
152 |
<programlisting> |
<programlisting> |
153 |
dgt:~/dejagnu.test$ ./configure |
dgt:~/dejagnu.test$ ./configure |
154 |
creating cache ./config.cache |
creating cache ./config.cache |
155 |
checking whether to enable maintainer-specific portions of Makefiles... no |
checking whether to enable maintainer-specific portions of Makefiles... no |
156 |
checking for a BSD compatible install... /usr/bin/install -c |
checking for a BSD compatible install... /usr/bin/install -c |
157 |
checking whether build environment is sane... yes |
checking whether build environment is sane... yes |
158 |
checking whether make sets ${MAKE}... yes |
checking whether make sets ${MAKE}... yes |
159 |
checking for working aclocal... found |
checking for working aclocal... found |
160 |
checking for working autoconf... found |
checking for working autoconf... found |
161 |
checking for working automake... found |
checking for working automake... found |
162 |
checking for working autoheader... found |
checking for working autoheader... found |
163 |
checking for working makeinfo... found |
checking for working makeinfo... found |
164 |
checking for gcc... gcc checking whether the C compiler (gcc ) works... yes |
checking for gcc... gcc checking whether the C compiler (gcc ) works... yes |
165 |
checking whether the C compiler (gcc ) is a cross-compiler... no |
checking whether the C compiler (gcc ) is a cross-compiler... no |
166 |
checking whether we are using GNU C... yes |
checking whether we are using GNU C... yes |
167 |
checking whether gcc accepts -g... yes |
checking whether gcc accepts -g... yes |
168 |
checking for a BSD compatible install... /usr/bin/install -c |
checking for a BSD compatible install... /usr/bin/install -c |
169 |
checking how to run the C preprocessor... gcc -E |
checking how to run the C preprocessor... gcc -E |
170 |
checking for stdlib.h... yes |
checking for stdlib.h... yes |
171 |
checking for strcmp... yes |
checking for strcmp... yes |
172 |
updating cache ./config.cache |
updating cache ./config.cache |
173 |
creating ./config.status |
creating ./config.status |
174 |
creating Makefile creating calc.h |
creating Makefile creating calc.h |
175 |
</programlisting> |
</programlisting> |
176 |
</example> |
</example> |
177 |
|
|
178 |
<para>If you are familiar with GNU software, |
<para>If you are familiar with GNU software, |
179 |
this output should not contain any surprise to you. |
this output should not contain any surprise to you. |
180 |
Any errors should be easy to fix for such a simple program.</para> |
Any errors should be easy to fix for such a simple program.</para> |
181 |
|
|
182 |
<para>Build the calc executable:</para> |
<para>Build the calc executable:</para> |
185 |
<title>Sample output building calc |
<title>Sample output building calc |
186 |
</title> |
</title> |
187 |
<programlisting> |
<programlisting> |
188 |
dgt:~/dejagnu.test$ make |
dgt:~/dejagnu.test$ make |
189 |
gcc -DHAVE_CONFIG_H -I. -I. -I. -g -O2 -c calc.c |
gcc -DHAVE_CONFIG_H -I. -I. -I. -g -O2 -c calc.c |
190 |
gcc -g -O2 -o calc calc.o |
gcc -g -O2 -o calc calc.o |
191 |
</programlisting> |
</programlisting> |
192 |
</example> |
</example> |
193 |
|
|
194 |
<para>You prepared a few files and then called some commands. |
<para>You prepared a few files and then called some commands. |
195 |
Respecting the right order assures a automatic and correctly compiled calc program. The following example resumes the correct order.</para> |
Respecting the right order assures a automatic and correctly compiled calc program. The following example resumes the correct order.</para> |
196 |
|
|
197 |
<example> |
<example> |
211 |
A sample session might look like this:</para> |
A sample session might look like this:</para> |
212 |
|
|
213 |
<programlisting> |
<programlisting> |
214 |
dgt:~/dejagnu.test$ ./calc |
dgt:~/dejagnu.test$ ./calc |
215 |
calc: version |
calc: version |
216 |
Version: 1.1 |
Version: 1.1 |
217 |
calc:<emphasis> </emphasis>add 3 4 |
calc:<emphasis> </emphasis>add 3 4 |
218 |
7 |
7 |
219 |
calc: multiply 3 4<emphasis> </emphasis> |
calc: multiply 3 4<emphasis> </emphasis> |
220 |
12 |
12 |
221 |
calc: multiply 2 4<emphasis> </emphasis> |
calc: multiply 2 4<emphasis> </emphasis> |
222 |
12 |
12 |
223 |
calc: quit |
calc: quit |
224 |
|
|
225 |
</programlisting> |
</programlisting> |
226 |
|
|
227 |
<para>Look at the intentional bug that 2 times 4 equals 12.</para> |
<para>Look at the intentional bug that 2 times 4 equals 12.</para> |
228 |
<para>The tests run by DejaGnu need a file called site.exp, |
<para>The tests run by DejaGnu need a file called site.exp, |
229 |
which is automatically generated if we call “make site.exp”. |
which is automatically generated if we call “make site.exp”. |
230 |
This was the purpose of the “AUTOMAKE_OPTIONS = dejagnu” in Makefile.am.</para> |
This was the purpose of the “AUTOMAKE_OPTIONS = dejagnu” in Makefile.am.</para> |
231 |
|
|
232 |
<example> |
<example> |
233 |
<title>Sample output generating a site.exp</title> |
<title>Sample output generating a site.exp</title> |
234 |
<programlisting> |
<programlisting> |
235 |
dgt: make site.exp |
dgt: make site.exp |
236 |
dgt:~/dejagnu.test$ make site.exp |
dgt:~/dejagnu.test$ make site.exp |
237 |
Making a new site.exp file... |
Making a new site.exp file... |
238 |
</programlisting> |
</programlisting> |
239 |
</example> |
</example> |
252 |
<title>Sample output of runtest in a configured directory</title> |
<title>Sample output of runtest in a configured directory</title> |
253 |
|
|
254 |
<programlisting> |
<programlisting> |
255 |
dgt:~/dejagnu.test$ make check |
dgt:~/dejagnu.test$ make check |
256 |
make check-DEJAGNU |
make check-DEJAGNU |
257 |
make[1]: Entering directory `/home/dgt/dejagnu.test' srcdir=`cd . && pwd`; export srcdir; \ |
make[1]: Entering directory `/home/dgt/dejagnu.test' srcdir=`cd . && pwd`; export srcdir; \ |
258 |
EXPECT=expect; export EXPECT; \ runtest=runtest; \ |
EXPECT=expect; export EXPECT; \ runtest=runtest; \ |
259 |
if /bin/sh -c "$runtest --version" > /dev/null 2>&1; then \ |
if /bin/sh -c "$runtest --version" > /dev/null 2>&1; then \ |
260 |
$runtest --tool calc CALC=`pwd`/calc --srcdir ./testsuite ; \ |
$runtest --tool calc CALC=`pwd`/calc --srcdir ./testsuite ; \ |
261 |
else echo "WARNING: could not find \`runtest'" 1>&2; :;\ |
else echo "WARNING: could not find \`runtest'" 1>&2; :;\ |
262 |
fi |
fi |
263 |
WARNING: Couldn't find the global config file. |
WARNING: Couldn't find the global config file. |
264 |
WARNING: Couldn't find tool init file |
WARNING: Couldn't find tool init file |
265 |
Test Run By dgt on Sun Nov 25 21:42:21 2001 |
Test Run By dgt on Sun Nov 25 21:42:21 2001 |
266 |
Native configuration is i586-pc-linux-gnu |
Native configuration is i586-pc-linux-gnu |
267 |
|
|
268 |
=== calc tests === |
=== calc tests === |
269 |
|
|
270 |
Schedule of variations: |
Schedule of variations: |
271 |
unix |
unix |
272 |
|
|
273 |
Running target unix |
Running target unix |
274 |
Using /usr/share/dejagnu/baseboards/unix.exp as board description file for target. |
Using /usr/share/dejagnu/baseboards/unix.exp as board description file for target. |
275 |
Using /usr/share/dejagnu/config/unix.exp as generic interface file for target. |
Using /usr/share/dejagnu/config/unix.exp as generic interface file for target. |
276 |
Using ./testsuite/config/unix.exp as tool-and-target-specific interface file. |
Using ./testsuite/config/unix.exp as tool-and-target-specific interface file. |
277 |
Running ./testsuite/calc.test/calc.exp ... |
Running ./testsuite/calc.test/calc.exp ... |
278 |
FAIL: multiply2 (bad match) |
FAIL: multiply2 (bad match) |
279 |
|
|
280 |
=== calc Summary === |
=== calc Summary === |
281 |
|
|
282 |
# of expected passes 5 |
# of expected passes 5 |
283 |
# of unexpected failures 1 |
# of unexpected failures 1 |
284 |
/home/Dgt/dejagnu.test/calc version Version: 1.1 |
/home/Dgt/dejagnu.test/calc version Version: 1.1 |
285 |
make[1]: *** [check-DEJAGNU] Fehler 1 |
make[1]: *** [check-DEJAGNU] Fehler 1 |
286 |
make[1]: Leaving directory `/home/Dgt/dejagnu.test' make: *** [check-am] Fehler 2 |
make[1]: Leaving directory `/home/Dgt/dejagnu.test' make: *** [check-am] Fehler 2 |
287 |
</programlisting> |
</programlisting> |
288 |
</example> |
</example> |
289 |
|
|
290 |
<para>Did you see the line “FAIL:“? The test cases for calc catch the bug in the calc.c file. Fix the error in calc.c later as the following examples assume a unchanged calc.c.</para> |
<para>Did you see the line “FAIL:“? The test cases for calc catch the bug in the calc.c file. Fix the error in calc.c later as the following examples assume a unchanged calc.c.</para> |
291 |
|
|
292 |
<para>Examine the output files calc.sum and calc.log. |
<para>Examine the output files calc.sum and calc.log. |
293 |
Try to understand the testcases written in ~/dejagnu.test/testsuite/calc.test/calc.exp. |
Try to understand the testcases written in ~/dejagnu.test/testsuite/calc.test/calc.exp. |
294 |
To understand Expect you might take a look at the book "Exploring Expect", |
To understand Expect you might take a look at the book "Exploring Expect", |
295 |
which is an excellent resource for learning and using Expect. (Pub: O'Reilly, ISBN 1-56592-090-2) |
which is an excellent resource for learning and using Expect. (Pub: O'Reilly, ISBN 1-56592-090-2) |
296 |
The book contains hundreds of examples and also includes a tutorial on Tcl. |
The book contains hundreds of examples and also includes a tutorial on Tcl. |
297 |
Exploring Expect is 602 pages long. </para> |
Exploring Expect is 602 pages long. </para> |
298 |
</sect2> |
</sect2> |
299 |
|
|
306 |
puts "I am ~/.dejagnurc" |
puts "I am ~/.dejagnurc" |
307 |
</programlisting> |
</programlisting> |
308 |
|
|
309 |
<para>Rerun make check. Test whether the output contains "I am ~/.dejagnurc". |
<para>Rerun make check. Test whether the output contains "I am ~/.dejagnurc". |
310 |
Create ~/my_dejagnu.exp and insert the following line:</para> |
Create ~/my_dejagnu.exp and insert the following line:</para> |
311 |
|
|
312 |
<programlisting> |
<programlisting> |
319 |
dgt:~/dejagnu.test$ export DEJAGNU=~/my_dejagnu.exp |
dgt:~/dejagnu.test$ export DEJAGNU=~/my_dejagnu.exp |
320 |
</programlisting> |
</programlisting> |
321 |
|
|
322 |
<para>Run “make check” again. The output should not contain |
<para>Run “make check” again. The output should not contain |
323 |
“WARNING: Couldn't find the global config file.”. |
“WARNING: Couldn't find the global config file.”. |
324 |
Create the sub-directory lib. Create the file “calc.exp” in it and insert the following line:</para> |
Create the sub-directory lib. Create the file “calc.exp” in it and insert the following line:</para> |
325 |
|
|
326 |
<programlisting> |
<programlisting> |
327 |
puts "I am lib/calc.exp" |
puts "I am lib/calc.exp" |
328 |
</programlisting> |
</programlisting> |
329 |
|
|
330 |
<para>The last warning “WARNING: Couldn't find tool init file” |
<para>The last warning “WARNING: Couldn't find tool init file” |
331 |
should not be part of the output of make check. |
should not be part of the output of make check. |
332 |
Create the directory ˜/boards. Create the file ˜/boards/standard.exp and insert the following line:</para> |
Create the directory ˜/boards. Create the file ˜/boards/standard.exp and insert the following line:</para> |
333 |
|
|
334 |
<programlisting> |
<programlisting> |
340 |
<example> |
<example> |
341 |
<title>Sample output of runtest with the usual configuration files</title> |
<title>Sample output of runtest with the usual configuration files</title> |
342 |
<programlisting> |
<programlisting> |
343 |
dgt:~/dejagnu.test$ make check |
dgt:~/dejagnu.test$ make check |
344 |
<...> |
<...> |
345 |
fi |
fi |
346 |
I am ~/.dejagnurc |
I am ~/.dejagnurc |
347 |
I am ~/my_dejagnu.exp |
I am ~/my_dejagnu.exp |
348 |
I am lib/calc.exp |
I am lib/calc.exp |
349 |
Test Run By dgt on Sun Nov 25 22:19:14 2001 |
Test Run By dgt on Sun Nov 25 22:19:14 2001 |
350 |
Native configuration is i586-pc-linux-gnu |
Native configuration is i586-pc-linux-gnu |
351 |
|
|
352 |
=== calc tests === |
=== calc tests === |
353 |
Using /home/Dgt/boards/standard.exp as standard board description\ |
Using /home/Dgt/boards/standard.exp as standard board description\ |
354 |
file for build. |
file for build. |
355 |
I am ~/boards/standard.exp |
I am ~/boards/standard.exp |
356 |
Using /home/Dgt/boards/standard.exp as standard board description\ |
Using /home/Dgt/boards/standard.exp as standard board description\ |
357 |
file for host. |
file for host. |
358 |
I am ~/boards/standard.exp |
I am ~/boards/standard.exp |
359 |
|
|
360 |
Schedule of variations: |
Schedule of variations: |
361 |
unix |
unix |
362 |
|
|
363 |
Running target unix |
Running target unix |
364 |
Using /home/Dgt/boards/standard.exp as standard board description\ |
Using /home/Dgt/boards/standard.exp as standard board description\ |
365 |
file for target. |
file for target. |
366 |
I am ~/boards/standard.exp |
I am ~/boards/standard.exp |
367 |
Using /usr/share/dejagnu/baseboards/unix.exp as board description file\ |
Using /usr/share/dejagnu/baseboards/unix.exp as board description file\ |
368 |
for target. |
for target. |
369 |
<...> |
<...> |
370 |
</programlisting> |
</programlisting> |
371 |
</example> |
</example> |
372 |
|
|
373 |
<para>It is up to you to decide when and where to use any of the above |
<para>It is up to you to decide when and where to use any of the above |
374 |
mentioned config files for customizing. |
mentioned config files for customizing. |
375 |
This chapters showed you where and in which order the different config |
This chapters showed you where and in which order the different config |
376 |
files are run.</para> |
files are run.</para> |
377 |
</sect2> |
</sect2> |
378 |
|
|
402 |
<title>Testing “Hello world” locally</title> |
<title>Testing “Hello world” locally</title> |
403 |
|
|
404 |
<para>This test checks, whether the built-in shell command “echo Hello world” |
<para>This test checks, whether the built-in shell command “echo Hello world” |
405 |
will really write “Hello world” on the console. |
will really write “Hello world” on the console. |
406 |
Create the file ~/dejagnu.test/testsuite/calc.test/local_echo.exp. |
Create the file ~/dejagnu.test/testsuite/calc.test/local_echo.exp. |
407 |
It should contain the following lines</para> |
It should contain the following lines</para> |
408 |
|
|
409 |
<example> |
<example> |
410 |
<title>A first (local) test case</title> |
<title>A first (local) test case</title> |
411 |
<programlisting> |
<programlisting> |
412 |
set test "Local Hello World" |
set test "Local Hello World" |
413 |
send "echo Hello World" |
send "echo Hello World" |
414 |
expect { |
expect { |
415 |
-re "Hello World" { pass "$test" } |
-re "Hello World" { pass "$test" } |
416 |
} |
} |
417 |
</programlisting> |
</programlisting> |
418 |
</example> |
</example> |
426 |
|
|
427 |
<para>Testing remote targets is a lot trickier especially if you are using an |
<para>Testing remote targets is a lot trickier especially if you are using an |
428 |
embedded target |
embedded target |
429 |
which has no built in support for things like a compiler, ftp server or a Bash-shell. |
which has no built in support for things like a compiler, ftp server or a Bash-shell. |
430 |
Before you can test calc on a remote target you have to acquire a few basics skills.</para> |
Before you can test calc on a remote target you have to acquire a few basics skills.</para> |
431 |
|
|
432 |
<sect2> |
<sect2> |
433 |
<title>Setup telnet to your own host</title> |
<title>Setup telnet to your own host</title> |
434 |
<para>The easiest remote host is usually the host you are working on. |
<para>The easiest remote host is usually the host you are working on. |
435 |
In this example we will use telnet to login in your own workstation. |
In this example we will use telnet to login in your own workstation. |
436 |
For security reason you should never have a telnet deamon running on |
For security reason you should never have a telnet deamon running on |
437 |
machine connected on the internet, as password and usernames are transmitted |
machine connected on the internet, as password and usernames are transmitted |
438 |
in clear text. |
in clear text. |
439 |
We assume you know how to setup your machine for a telnet daemon.</para> |
We assume you know how to setup your machine for a telnet daemon.</para> |
440 |
|
|
441 |
<para>Next try whether you may login in your own host by issuing the |
<para>Next try whether you may login in your own host by issuing the |
442 |
command “telnet localhost.1”. In order to be able to |
command “telnet localhost.1”. In order to be able to |
443 |
distinguish between a normal session an a telnet login add the following lines to /home/dgt/.bashrc.</para> |
distinguish between a normal session an a telnet login add the following lines to /home/dgt/.bashrc.</para> |
444 |
|
|
445 |
<programlisting> |
<programlisting> |
446 |
if [ "$REMOTEHOST" ] |
if [ "$REMOTEHOST" ] |
447 |
then |
then |
448 |
PS1='remote:\w\$ ' |
PS1='remote:\w\$ ' |
449 |
fi |
fi |
450 |
</programlisting> |
</programlisting> |
451 |
|
|
455 |
<title>Sample log of a telnet login to localhost</title> |
<title>Sample log of a telnet login to localhost</title> |
456 |
<programlisting> |
<programlisting> |
457 |
dgt:~/dejagnu.test$ telnet localhost |
dgt:~/dejagnu.test$ telnet localhost |
458 |
Trying 127.0.0.1... |
Trying 127.0.0.1... |
459 |
Connected to 127.0.0.1. |
Connected to 127.0.0.1. |
460 |
Escape character is '^]'. |
Escape character is '^]'. |
461 |
Debian GNU/Linux testing/unstable Linux |
Debian GNU/Linux testing/unstable Linux |
462 |
K6Linux login: dgt |
K6Linux login: dgt |
463 |
Password: |
Password: |
464 |
Last login: Sun Nov 25 22:46:34 2001 from localhost on pts/4 |
Last login: Sun Nov 25 22:46:34 2001 from localhost on pts/4 |
465 |
Linux K6Linux 2.4.14 #1 Fre Nov 16 19:28:25 CET 2001 i586 unknown |
Linux K6Linux 2.4.14 #1 Fre Nov 16 19:28:25 CET 2001 i586 unknown |
466 |
No mail. |
No mail. |
467 |
remote:~$ exit |
remote:~$ exit |
468 |
logout |
logout |
469 |
Connection closed by foreign host. |
Connection closed by foreign host. |
470 |
</programlisting> |
</programlisting> |
471 |
</example> |
</example> |
473 |
|
|
474 |
<sect2> |
<sect2> |
475 |
<title>A test case for login via telnet</title> |
<title>A test case for login via telnet</title> |
476 |
<para>In order to define a correct setup we have add a line containing |
<para>In order to define a correct setup we have add a line containing |
477 |
“set target unix” either to ~/.dejagnurc or to ~/my_dejagnu.exp. |
“set target unix” either to ~/.dejagnurc or to ~/my_dejagnu.exp. |
478 |
In ~/boards/standard.exp add the following four lines to define a few patterns for the DejaGnu telnet login procedure.</para> |
In ~/boards/standard.exp add the following four lines to define a few patterns for the DejaGnu telnet login procedure.</para> |
479 |
|
|
480 |
<example> |
<example> |
481 |
<title>Defining a remote target board</title> |
<title>Defining a remote target board</title> |
482 |
<programlisting> |
<programlisting> |
483 |
set_board_info shell_prompt "remote:" |
set_board_info shell_prompt "remote:" |
484 |
set_board_info telnet_username "dgt" |
set_board_info telnet_username "dgt" |
485 |
set_board_info telnet_password "top_secret" |
set_board_info telnet_password "top_secret" |
486 |
set_board_info hostname "localhost" |
set_board_info hostname "localhost" |
487 |
|
|
488 |
</programlisting> |
</programlisting> |
489 |
</example> |
</example> |
490 |
|
|
491 |
<para>As DejaGnu will be parsing the telnet session output for some well |
<para>As DejaGnu will be parsing the telnet session output for some well |
492 |
known pattern the output there are a lot of things that can go wrong. |
known pattern the output there are a lot of things that can go wrong. |
493 |
If you have any problems verify your setup:</para> |
If you have any problems verify your setup:</para> |
494 |
<itemizedlist> |
<itemizedlist> |
495 |
|
|
506 |
<para>The LANG environment variable must be either empty or set to “C”. </para></listitem> |
<para>The LANG environment variable must be either empty or set to “C”. </para></listitem> |
507 |
|
|
508 |
</itemizedlist> |
</itemizedlist> |
509 |
<para>To test the login via telnet write a sample test case. |
<para>To test the login via telnet write a sample test case. |
510 |
Create the file ~/dejagnu.test/testsuite/calc.test/remote_echo.exp and |
Create the file ~/dejagnu.test/testsuite/calc.test/remote_echo.exp and |
511 |
add the following few lines:</para> |
add the following few lines:</para> |
512 |
|
|
513 |
<example> |
<example> |
514 |
<title>DejaGnu script for logging in into a remote target</title> |
<title>DejaGnu script for logging in into a remote target</title> |
515 |
<programlisting> |
<programlisting> |
516 |
puts "this is remote_echo.exp target for $target " |
puts "this is remote_echo.exp target for $target " |
517 |
target_info $target |
target_info $target |
518 |
#set verbose 9 |
#set verbose 9 |
519 |
set shell_id [remote_open $target] |
set shell_id [remote_open $target] |
520 |
set test "Remote login to $target" |
set test "Remote login to $target" |
521 |
#set verbose 0 |
#set verbose 0 |
522 |
puts "Spawn id for remote shell is $shell_id" |
puts "Spawn id for remote shell is $shell_id" |
523 |
if { $shell_id > 0 } { |
if { $shell_id > 0 } { |
524 |
pass "$test" |
pass "$test" |
525 |
} else { |
} else { |
526 |
fail "Remote open to $target" |
fail "Remote open to $target" |
527 |
} |
} |
528 |
</programlisting> |
</programlisting> |
529 |
</example> |
</example> |
531 |
<para>In the runtest output you should find something like:</para> |
<para>In the runtest output you should find something like:</para> |
532 |
|
|
533 |
<programlisting> |
<programlisting> |
534 |
Running ./testsuite/calc.test/local_echo.exp ... |
Running ./testsuite/calc.test/local_echo.exp ... |
535 |
Running ./testsuite/calc.test/remote_echoo.exp ... |
Running ./testsuite/calc.test/remote_echoo.exp ... |
536 |
this is remote_echo.exp target is unix |
this is remote_echo.exp target is unix |
537 |
Spawn id for remote shell is exp7 |
Spawn id for remote shell is exp7 |
538 |
</programlisting> |
</programlisting> |
539 |
|
|
543 |
<sect2> |
<sect2> |
544 |
<title>Remote testing “Hello world”</title> |
<title>Remote testing “Hello world”</title> |
545 |
|
|
546 |
<para>Next you will transform the above “hello world” example to |
<para>Next you will transform the above “hello world” example to |
547 |
its remote equivalent. |
its remote equivalent. |
548 |
This can be done by adding the following lines to our file remote_echo.exp.</para> |
This can be done by adding the following lines to our file remote_echo.exp.</para> |
549 |
|
|
550 |
<example> |
<example> |
551 |
<title>A first (local) remote "Hello world" test</title> |
<title>A first (local) remote "Hello world" test</title> |
552 |
<programlisting> |
<programlisting> |
553 |
set test "Remote_send Hello World" |
set test "Remote_send Hello World" |
554 |
set status [remote_send $target "echo \"Hello\" \"World\"\n" ] |
set status [remote_send $target "echo \"Hello\" \"World\"\n" ] |
555 |
pass "$test" |
pass "$test" |
556 |
set test "Remote_expect Hello World" |
set test "Remote_expect Hello World" |
557 |
remote_expect $target 5 { |
remote_expect $target 5 { |
558 |
-re "Hello World" { pass "$test" } |
-re "Hello World" { pass "$test" } |
559 |
} |
} |
560 |
</programlisting> |
</programlisting> |
561 |
</example> |
</example> |
565 |
|
|
566 |
<para>Have a look at the procedures in /usr/share/dejagnu/remote.exp to have an overview of the offered procedures and their features. </para> |
<para>Have a look at the procedures in /usr/share/dejagnu/remote.exp to have an overview of the offered procedures and their features. </para> |
567 |
|
|
568 |
<para>Now setup a real target. |
<para>Now setup a real target. |
569 |
In the following example we assume as target a PowerBook running Debian. |
In the following example we assume as target a PowerBook running Debian. |
570 |
As above add a test user "dgt", install telnet and FTP servers. |
As above add a test user "dgt", install telnet and FTP servers. |
571 |
In order to distinguish it from the host add the line |
In order to distinguish it from the host add the line |
572 |
<programlisting>PS1='test:>'</programlisting> to /home/dgt/.bash_profile. |
<programlisting>PS1='test:>'</programlisting> to /home/dgt/.bash_profile. |
573 |
Also add a corresponding entry "powerbook" to /etc/hosts and verify that you |
Also add a corresponding entry "powerbook" to /etc/hosts and verify that you |
574 |
are able to ping, telnet and ftp to the target "powerbook".</para> |
are able to ping, telnet and ftp to the target "powerbook".</para> |
575 |
|
|
576 |
<para>In order to let runtest run its test on the "powerbook" target change the following lines in ~/boards/standard.exp:</para> |
<para>In order to let runtest run its test on the "powerbook" target change the following lines in ~/boards/standard.exp:</para> |
615 |
</programlisting> |
</programlisting> |
616 |
</example> |
</example> |
617 |
|
|
618 |
<para>After running runtest again, check whether the file dejagnu2 exists on the target. |
<para>After running runtest again, check whether the file dejagnu2 exists on the target. |
619 |
|
|
620 |
This example will only work if the rcp command works with your target. |
This example will only work if the rcp command works with your target. |
621 |
|
|
622 |
If you have a working FTP-server on the target you can use it by adding the |
If you have a working FTP-server on the target you can use it by adding the |
623 |
following lines to ~/boards/standard.exp:</para> |
following lines to ~/boards/standard.exp:</para> |
624 |
|
|
625 |
<example> |
<example> |
637 |
<title>Preparing for crosscompilation</title> |
<title>Preparing for crosscompilation</title> |
638 |
|
|
639 |
<para>For crosscompiling you need working binutils, gcc and a base library like |
<para>For crosscompiling you need working binutils, gcc and a base library like |
640 |
libc or glib for your target. |
libc or glib for your target. |
641 |
It is beyond the scope of this document to describe how to get it working. |
It is beyond the scope of this document to describe how to get it working. |
642 |
The following examples assume a cross compiler for PowerPC which is called linux-powerpc-gcc. |
The following examples assume a cross compiler for PowerPC which is called linux-powerpc-gcc. |
643 |
</para> |
</para> |
644 |
|
|
646 |
|
|
647 |
<programlisting> |
<programlisting> |
648 |
AM_CONFIG_HEADER(calc.h) |
AM_CONFIG_HEADER(calc.h) |
649 |
AC_CANONICAL_TARGET([]) |
AC_CANONICAL_TARGET([]) |
650 |
AM_INIT_AUTOMAKE(calc, 1.1) |
AM_INIT_AUTOMAKE(calc, 1.1) |
651 |
</programlisting> |
</programlisting> |
652 |
|
|
653 |
<para>You need to run automake 2.5 or later. |
<para>You need to run automake 2.5 or later. |
654 |
Depending on your installation calling autoconf2.5 instead of autoconf is not needed. |
Depending on your installation calling autoconf2.5 instead of autoconf is not needed. |
655 |
The sequence to regenerate all files is:</para> |
The sequence to regenerate all files is:</para> |
656 |
|
|
657 |
<example> |
<example> |
660 |
$ autoconf2.5 |
$ autoconf2.5 |
661 |
$ autoheader |
$ autoheader |
662 |
$ automake |
$ automake |
663 |
$ ./configure --host=powerpc-linux --target=powerpc-linux |
$ ./configure --host=powerpc-linux --target=powerpc-linux |
664 |
configure: WARNING: If you wanted to set the --build type, don't use --host. |
configure: WARNING: If you wanted to set the --build type, don't use --host. |
665 |
If a cross compiler is detected then cross compile mode will be used. |
If a cross compiler is detected then cross compile mode will be used. |
666 |
checking build system type... ./config.guess: ./config.guess: No such file or directory |
checking build system type... ./config.guess: ./config.guess: No such file or directory |
667 |
configure: error: cannot guess build type; you must specify one |
configure: error: cannot guess build type; you must specify one |
668 |
$ cp /usr/share/automake/config.guess . |
$ cp /usr/share/automake/config.guess . |
669 |
$ ./configure --host=powerpc-linux --target=powerpc-linux |
$ ./configure --host=powerpc-linux --target=powerpc-linux |
670 |
configure: WARNING: If you wanted to set the --build type, don't use --host. |
configure: WARNING: If you wanted to set the --build type, don't use --host. |
671 |
If a cross compiler is detected then cross compile mode will be used. \ |
If a cross compiler is detected then cross compile mode will be used. \ |
672 |
checking build system type... i586-pc-linux-gnu |
checking build system type... i586-pc-linux-gnu |
685 |
<para>Everything should be ready to recompile for the target:</para> |
<para>Everything should be ready to recompile for the target:</para> |
686 |
|
|
687 |
<programlisting>$ make |
<programlisting>$ make |
688 |
powerpc-linux-gcc -DHAVE_CONFIG_H -I. -I. -I. -g -O2 -c calc.c |
powerpc-linux-gcc -DHAVE_CONFIG_H -I. -I. -I. -g -O2 -c calc.c |
689 |
powerpc-linux-gcc -g -O2 -o calc calc.o |
powerpc-linux-gcc -g -O2 -o calc calc.o |
690 |
|
|
691 |
</programlisting> |
</programlisting> |
755 |
<filename>Makefile</filename>.</para> |
<filename>Makefile</filename>.</para> |
756 |
|
|
757 |
</sect1> |
</sect1> |
758 |
|
|
759 |
<sect1 id=runtest xreflabel="Runtest"> |
<sect1 id=runtest xreflabel="Runtest"> |
760 |
<title>Runtest</title> |
<title>Runtest</title> |
761 |
|
|
792 |
case to determine whether you should amend it to stop expecting |
case to determine whether you should amend it to stop expecting |
793 |
failure.</para></listitem> |
failure.</para></listitem> |
794 |
</varlistentry> |
</varlistentry> |
795 |
|
|
796 |
<varlistentry> |
<varlistentry> |
797 |
<term>FAIL</term> |
<term>FAIL</term> |
798 |
<listitem><para>A test failed, although it was expected to succeed. |
<listitem><para>A test failed, although it was expected to succeed. |
799 |
This may indicate regress; inspect the test case and the failing |
This may indicate regress; inspect the test case and the failing |
800 |
software to ocate the bug.</para></listitem> |
software to ocate the bug.</para></listitem> |
801 |
</varlistentry> |
</varlistentry> |
802 |
|
|
803 |
<varlistentry> |
<varlistentry> |
804 |
<term>XFAIL</term> |
<term>XFAIL</term> |
805 |
<listitem><para>A test failed, but it was expected to fail. This |
<listitem><para>A test failed, but it was expected to fail. This |
808 |
by the test, the outcome is <emphasis>UNSUPPORTED</emphasis> |
by the test, the outcome is <emphasis>UNSUPPORTED</emphasis> |
809 |
instead.</para></listitem> |
instead.</para></listitem> |
810 |
</varlistentry> |
</varlistentry> |
811 |
|
|
812 |
<varlistentry> |
<varlistentry> |
813 |
<term>UNRESOLVED</term> |
<term>UNRESOLVED</term> |
814 |
<listitem><para>Output from a test requires manual inspection; the |
<listitem><para>Output from a test requires manual inspection; the |
816 |
example, your tests can report this outcome is when a test does not |
example, your tests can report this outcome is when a test does not |
817 |
complete as expected.</para></listitem> |
complete as expected.</para></listitem> |
818 |
</varlistentry> |
</varlistentry> |
819 |
|
|
820 |
<varlistentry> |
<varlistentry> |
821 |
<term>UNTESTED</term> |
<term>UNTESTED</term> |
822 |
<listitem><para>A test case is not yet complete, and in particular |
<listitem><para>A test case is not yet complete, and in particular |
825 |
``tests'' that note explicitly the absence of a real test case for a |
``tests'' that note explicitly the absence of a real test case for a |
826 |
particular property.</para></listitem> |
particular property.</para></listitem> |
827 |
</varlistentry> |
</varlistentry> |
828 |
|
|
829 |
<varlistentry> |
<varlistentry> |
830 |
<term>UNSUPPORTED</term> |
<term>UNSUPPORTED</term> |
831 |
<listitem><para>A test depends on a conditionally available feature |
<listitem><para>A test depends on a conditionally available feature |
1061 |
test. Executable tool names (and paths) are recorded in |
test. Executable tool names (and paths) are recorded in |
1062 |
<filename>site.exp</filename> and you can override them by specifying |
<filename>site.exp</filename> and you can override them by specifying |
1063 |
Tcl variables on the command line.</para> |
Tcl variables on the command line.</para> |
1064 |
|
|
1065 |
<para>For example, including "<option>--tool</option> gcc" on the |
<para>For example, including "<option>--tool</option> gcc" on the |
1066 |
<emphasis>runtest</emphasis> command line runs tests from all test |
<emphasis>runtest</emphasis> command line runs tests from all test |
1067 |
subdirectories whose names match <filename>gcc.*</filename>, and uses |
subdirectories whose names match <filename>gcc.*</filename>, and uses |
1072 |
<emphasis>GCC=binname</emphasis> on the <emphasis>runtest</emphasis> |
<emphasis>GCC=binname</emphasis> on the <emphasis>runtest</emphasis> |
1073 |
command line.</para></listitem> |
command line.</para></listitem> |
1074 |
</varlistentry> |
</varlistentry> |
1075 |
|
|
1076 |
<varlistentry> |
<varlistentry> |
1077 |
<term><option>--tool_exec [name]</option></term> |
<term><option>--tool_exec [name]</option></term> |
1078 |
<listitem><para>The path to the tool executable to |
<listitem><para>The path to the tool executable to |
1095 |
file, but not in the summary (<filename>*.sum</filename>) log |
file, but not in the summary (<filename>*.sum</filename>) log |
1096 |
file.</para></listitem> |
file.</para></listitem> |
1097 |
</varlistentry> |
</varlistentry> |
1098 |
|
|
1099 |
<varlistentry> |
<varlistentry> |
1100 |
<term><option>--version</option> (-V)</term> |
<term><option>--version</option> (-V)</term> |
1101 |
<listitem><para>Prints out the version numbers of DejaGnu, |
<listitem><para>Prints out the version numbers of DejaGnu, |
1130 |
</varlistentry> |
</varlistentry> |
1131 |
|
|
1132 |
<varlistentry> |
<varlistentry> |
1133 |
<term><filename>testfile</filename>.exp="testfile1 ..."</term> |
<term><filename>testfile</filename>.exp="testfile1 ..."</term> |
1134 |
<listitem><para>Specify a subset of tests in a suite to run. For |
<listitem><para>Specify a subset of tests in a suite to run. For |
1135 |
compiler or assembler tests, which often use a single |
compiler or assembler tests, which often use a single |
1136 |
<emphasis>.exp</emphasis> script covering many different source |
<emphasis>.exp</emphasis> script covering many different source |
1185 |
/usr/latest/bin/gdb version 4.14.4 -nx |
/usr/latest/bin/gdb version 4.14.4 -nx |
1186 |
</screen> |
</screen> |
1187 |
|
|
1188 |
<para>You can use the option <emphasis>--srcdir</emphasis> to point to |
<para>You can use the option <emphasis>--srcdir</emphasis> to point to |
1189 |
some other directory containing a collection of tests:</para> |
some other directory containing a collection of tests:</para> |
1190 |
|
|
1191 |
<screen> |
<screen> |
1211 |
<filename>gdb.sum</filename> and |
<filename>gdb.sum</filename> and |
1212 |
<filename>gdb.log</filename>.)</para> |
<filename>gdb.log</filename>.)</para> |
1213 |
</sect2> |
</sect2> |
1214 |
</sect1> |
</sect1> |
1215 |
|
|
1216 |
<sect1 id=outputfiles xreflabel="Output Files"> |
<sect1 id=outputfiles xreflabel="Output Files"> |
1217 |
|
|
1270 |
FAIL: help run "r" abbreviation |
FAIL: help run "r" abbreviation |
1271 |
Running ./gdb.t10/crossload.exp ... |
Running ./gdb.t10/crossload.exp ... |
1272 |
PASS: m68k-elf (elf-big) explicit format; loaded |
PASS: m68k-elf (elf-big) explicit format; loaded |
1273 |
XFAIL: mips-ecoff (ecoff-bigmips) "ptype v_signed_char" signed C types |
XFAIL: mips-ecoff (ecoff-bigmips) "ptype v_signed_char" signed C types |
1274 |
=== gdb Summary === |
=== gdb Summary === |
1275 |
# of expected passes 5 |
# of expected passes 5 |
1276 |
# of expected failures 1 |
# of expected failures 1 |
1394 |
{command}? no |
{command}? no |
1395 |
break gdbme.c:34 |
break gdbme.c:34 |
1396 |
Breakpoint 8 at 0x23d8: file gdbme.c, line 34. |
Breakpoint 8 at 0x23d8: file gdbme.c, line 34. |
1397 |
(gdb) expect: does {break gdbme.c:34\r\nBreakpoint 8 at 0x23d8: |
(gdb) expect: does {break gdbme.c:34\r\nBreakpoint 8 at 0x23d8: |
1398 |
file gdbme.c, line 34.\r\n(gdb) } (spawn_id 6) match pattern |
file gdbme.c, line 34.\r\n(gdb) } (spawn_id 6) match pattern |
1399 |
{Breakpoint.*at.* file gdbme.c, line 34.*\(gdb\) $}? yes |
{Breakpoint.*at.* file gdbme.c, line 34.*\(gdb\) $}? yes |
1400 |
expect: set expect_out(0,start) {18} |
expect: set expect_out(0,start) {18} |
1417 |
<listitem><para>Empty output for the first attempted match. The |
<listitem><para>Empty output for the first attempted match. The |
1418 |
first set of attempted matches shown ran against the output |
first set of attempted matches shown ran against the output |
1419 |
<emphasis>{}</emphasis> --- that is, no |
<emphasis>{}</emphasis> --- that is, no |
1420 |
output. <productname>Expect</productname> begins |
output. <productname>Expect</productname> begins |
1421 |
attempting to match the patterns supplied immediately; often, |
attempting to match the patterns supplied immediately; often, |
1422 |
the first pass is against incomplete output (or completely |
the first pass is against incomplete output (or completely |
1423 |
before all output, as in this case).</para></listitem> |
before all output, as in this case).</para></listitem> |
1456 |
</sect2> |
</sect2> |
1457 |
</sect1> |
</sect1> |
1458 |
</chapter> |
</chapter> |
1459 |
|
|
1460 |
<chapter id=Customizing xreflabel="Customizing DejaGnu"> |
<chapter id=Customizing xreflabel="Customizing DejaGnu"> |
1461 |
<title>Customizing DejaGnu</title> |
<title>Customizing DejaGnu</title> |
1462 |
|
|
1463 |
<para>The site configuration file, <filename>site.exp</filename>, |
<para>The site configuration file, <filename>site.exp</filename>, |
1464 |
captures configuration-dependent values and propagates them to the |
captures configuration-dependent values and propagates them to the |
1465 |
DejaGnu test environment using Tcl variables. This ties the |
DejaGnu test environment using Tcl variables. This ties the |
1494 |
either rely on and use, or override, any of the global values from |
either rely on and use, or override, any of the global values from |
1495 |
the global <filename>site.exp</filename> file.</para> |
the global <filename>site.exp</filename> file.</para> |
1496 |
|
|
1497 |
<para>You can usually generate or update the testsuite's local |
<para>You can usually generate or update the testsuite's local |
1498 |
<filename>site.exp</filename> by typing <command>make |
<filename>site.exp</filename> by typing <command>make |
1499 |
site.exp</command> in the testsuite directory, after the test |
site.exp</command> in the testsuite directory, after the test |
1500 |
suite is configured.</para> |
suite is configured.</para> |
1513 |
|
|
1514 |
<sect1 id=local xreflabel="Local Config File"> |
<sect1 id=local xreflabel="Local Config File"> |
1515 |
<title>Local Config File</title> |
<title>Local Config File</title> |
1516 |
|
|
1517 |
<para>It is usually more convenient to keep these <emphasis>manual |
<para>It is usually more convenient to keep these <emphasis>manual |
1518 |
overrides</emphasis> in the <filename>site.exp</filename> |
overrides</emphasis> in the <filename>site.exp</filename> |
1519 |
local to each test directory, rather than in the global |
local to each test directory, rather than in the global |
1579 |
<example> |
<example> |
1580 |
<title>Local Config File</title> |
<title>Local Config File</title> |
1581 |
|
|
1582 |
<programlisting> |
<programlisting> |
1583 |
## these variables are automatically generated by make ## |
## these variables are automatically generated by make ## |
1584 |
# Do not edit here. If you wish to override these values |
# Do not edit here. If you wish to override these values |
1585 |
# add them to the last section |
# add them to the last section |
1594 |
set tmpdir /build/devo-builds/i586-pc-linux-gnulibc1/gcc/testsuite |
set tmpdir /build/devo-builds/i586-pc-linux-gnulibc1/gcc/testsuite |
1595 |
set srcdir "${srcdir}/testsuite" |
set srcdir "${srcdir}/testsuite" |
1596 |
## All variables above are generated by configure. Do Not Edit ## |
## All variables above are generated by configure. Do Not Edit ## |
1597 |
|
|
1598 |
</programlisting> |
</programlisting> |
1599 |
</example> |
</example> |
1600 |
|
|
1601 |
<para>This file defines the required fields for a local config |
<para>This file defines the required fields for a local config |
1608 |
</sect1> |
</sect1> |
1609 |
<sect1 id=global xreflabel="Global Config File"> |
<sect1 id=global xreflabel="Global Config File"> |
1610 |
<title>Global Config File</title> |
<title>Global Config File</title> |
1611 |
|
|
1612 |
<para>The master config file is where all the target specific |
<para>The master config file is where all the target specific |
1613 |
config variables for a whole site get set. The idea is |
config variables for a whole site get set. The idea is |
1614 |
that for a centralized testing lab where people have to share a |
that for a centralized testing lab where people have to share a |
1686 |
|
|
1687 |
<sect1 id=boardconfig xreflabel="Board Config File"> |
<sect1 id=boardconfig xreflabel="Board Config File"> |
1688 |
<title>Board Config File</title> |
<title>Board Config File</title> |
1689 |
|
|
1690 |
<para>The board config file is where board specfic config data |
<para>The board config file is where board specfic config data |
1691 |
is stored. A board config file contains all the higher-level |
is stored. A board config file contains all the higher-level |
1692 |
configuration settings. There is a rough inheritance scheme, where it is |
configuration settings. There is a rough inheritance scheme, where it is |
1896 |
|
|
1897 |
<sect1 id=configfile xreflabel="Config File Values"> |
<sect1 id=configfile xreflabel="Config File Values"> |
1898 |
<title>Config File Values</title> |
<title>Config File Values</title> |
1899 |
|
|
1900 |
<para>DejaGnu uses a named array in Tcl to hold all the info for |
<para>DejaGnu uses a named array in Tcl to hold all the info for |
1901 |
each machine. In the case of a canadian cross, this means host |
each machine. In the case of a canadian cross, this means host |
1902 |
information as well as target information. The named array is |
information as well as target information. The named array is |
1911 |
variables captured in the first section, but also specify |
variables captured in the first section, but also specify |
1912 |
default values for all on the <command>runtest</command> |
default values for all on the <command>runtest</command> |
1913 |
command line options. Save for <option>--debug</option>, |
command line options. Save for <option>--debug</option>, |
1914 |
<option>--help</option>, and <option>--version</option>, each |
<option>--help</option>, and <option>--version</option>, each |
1915 |
command line option has an associated Tcl variable. Use the |
command line option has an associated Tcl variable. Use the |
1916 |
Tcl <command>set</command> command to specify a new default |
Tcl <command>set</command> command to specify a new default |
1917 |
value (as for the configuration variables). The following |
value (as for the configuration variables). The following |
2032 |
|
|
2033 |
<sect2 id=personal xreflabel="Personal Config File"> |
<sect2 id=personal xreflabel="Personal Config File"> |
2034 |
<title>Personal Config File</title> |
<title>Personal Config File</title> |
2035 |
|
|
2036 |
<para>The personal config file is used to customize |
<para>The personal config file is used to customize |
2037 |
<command>runtest's</command> behaviour for each person. It's |
<command>runtest's</command> behaviour for each person. It's |
2038 |
typically used to set the user prefered setting for verbosity, |
typically used to set the user prefered setting for verbosity, |
2063 |
</sect1> |
</sect1> |
2064 |
|
|
2065 |
</chapter> |
</chapter> |
2066 |
|
|
2067 |
<chapter id=Extending xreflabel="Extending DejaGnu"> |
<chapter id=Extending xreflabel="Extending DejaGnu"> |
2068 |
<title>Extending DejaGnu</title> |
<title>Extending DejaGnu</title> |
2069 |
|
|
2070 |
<sect1 id=addsuite xreflabel="Adding a new Testsuite"> |
<sect1 id=addsuite xreflabel="Adding a new Testsuite"> |
2071 |
<title>Adding A New Testsuite</title> |
<title>Adding A New Testsuite</title> |
2072 |
|
|
2073 |
<para>The testsuite for a new tool should always be located in that tools |
<para>The testsuite for a new tool should always be located in that tools |
2074 |
source directory. DejaGnu require the directory be named |
source directory. DejaGnu require the directory be named |
2075 |
<filename>testsuite</filename>. Under this directory, the test cases go |
<filename>testsuite</filename>. Under this directory, the test cases go |
2081 |
|
|
2082 |
<sect1 id=addtool xreflabel="Adding A New Tool"> |
<sect1 id=addtool xreflabel="Adding A New Tool"> |
2083 |
<title>Adding A New Tool</title> |
<title>Adding A New Tool</title> |
2084 |
|
|
2085 |
<para>In general, the best way to learn how to write (code or even prose) |
<para>In general, the best way to learn how to write (code or even prose) |
2086 |
is to read something similar. This principle applies to test cases and |
is to read something similar. This principle applies to test cases and |
2087 |
to testsuites. Unfortunately, well-established testsuites have a way |
to testsuites. Unfortunately, well-established testsuites have a way |
2109 |
steps to begin building a testsuite for a program example.</para> |
steps to begin building a testsuite for a program example.</para> |
2110 |
|
|
2111 |
<itemizedlist mark=bullet> |
<itemizedlist mark=bullet> |
2112 |
|
|
2113 |
<listitem><para>Create or select a directory to contain your new |
<listitem><para>Create or select a directory to contain your new |
2114 |
collection of tests. Change into that directory (shown here as |
collection of tests. Change into that directory (shown here as |
2115 |
<filename>testsuite</filename>):</para> |
<filename>testsuite</filename>):</para> |
2116 |
|
|
2117 |
<para>Create a <filename>configure.in</filename> file in this directory, |
<para>Create a <filename>configure.in</filename> file in this directory, |
2118 |
to control configuration-dependent choices for your tests. So far as |
to control configuration-dependent choices for your tests. So far as |
2119 |
DejaGnu is concerned, the important thing is to set a value for the |
DejaGnu is concerned, the important thing is to set a value for the |
2164 |
# The flags to pass to runtest |
# The flags to pass to runtest |
2165 |
RUNTESTFLAGS = |
RUNTESTFLAGS = |
2166 |
|
|
2167 |
# Execute the tests |
# Execute the tests |
2168 |
check: site.exp all |
check: site.exp all |
2169 |
$(RUNTEST) $(RUNTESTFLAGS) \ |
$(RUNTEST) $(RUNTESTFLAGS) \ |
2170 |
--tool <symbol>${example}</symbol> --srcdir $(srcdir) |
--tool <symbol>${example}</symbol> --srcdir $(srcdir) |
2171 |
|
|
2172 |
# Make the local config file |
# Make the local config file |
2173 |
site.exp: ./config.status Makefile |
site.exp: ./config.status Makefile |
2206 |
</programlisting> |
</programlisting> |
2207 |
</example> |
</example> |
2208 |
</listitem> |
</listitem> |
2209 |
|
|
2210 |
<listitem><para>Create a directory (in <filename>testsuite</filename>) |
<listitem><para>Create a directory (in <filename>testsuite</filename>) |
2211 |
called <filename>config</filename>. Make a <emphasis>Tool Init |
called <filename>config</filename>. Make a <emphasis>Tool Init |
2212 |
File</emphasis> in this directory. Its name must start with the |
File</emphasis> in this directory. Its name must start with the |
2223 |
|
|
2224 |
<example> |
<example> |
2225 |
<title>Simple Batch Program Tool Init File</title> |
<title>Simple Batch Program Tool Init File</title> |
2226 |
|
|
2227 |
<programlisting> |
<programlisting> |
2228 |
|
|
2229 |
proc foo_exit {} {} |
proc foo_exit {} {} |
2230 |
proc foo_version {} {} |
proc foo_version {} {} |
2231 |
|
|
2321 |
|
|
2322 |
<sect1 id=addtarget xreflabel="Adding A New Target"> |
<sect1 id=addtarget xreflabel="Adding A New Target"> |
2323 |
<title>Adding A New Target</title> |
<title>Adding A New Target</title> |
2324 |
|
|
2325 |
<para>DejaGnu has some additional requirements for target support, beyond |
<para>DejaGnu has some additional requirements for target support, beyond |
2326 |
the general-purpose provisions of configure. DejaGnu must actively |
the general-purpose provisions of configure. DejaGnu must actively |
2327 |
communicate with the target, rather than simply generating or managing |
communicate with the target, rather than simply generating or managing |
2365 |
|
|
2366 |
<sect1 id=addboard xreflabel="Adding A New Board"> |
<sect1 id=addboard xreflabel="Adding A New Board"> |
2367 |
<title>Adding A New Board</title> |
<title>Adding A New Board</title> |
2368 |
|
|
2369 |
<para>Adding a new board consists of creating a new board config |
<para>Adding a new board consists of creating a new board config |
2370 |
file. Examples are in |
file. Examples are in |
2371 |
<filename>dejagnu/baseboards</filename>. Usually to make a new |
<filename>dejagnu/baseboards</filename>. Usually to make a new |
2386 |
contains other procedures used for a certain class of target. The |
contains other procedures used for a certain class of target. The |
2387 |
board description file is where the board specfic settings go. Commonly |
board description file is where the board specfic settings go. Commonly |
2388 |
there are similar target environments with just different |
there are similar target environments with just different |
2389 |
processors.</para> |
processors.</para> |
2390 |
|
|
2391 |
<example> |
<example> |
2392 |
<title>Testing a New Board Config File</title> |
<title>Testing a New Board Config File</title> |
2403 |
testing in the build tree, because the executables to be tested |
testing in the build tree, because the executables to be tested |
2404 |
are in the same tree as the new dejagnu files. The helper |
are in the same tree as the new dejagnu files. The helper |
2405 |
procedures are the ones in square braces |
procedures are the ones in square braces |
2406 |
<emphasis>[]</emphasis>, which is the Tcl execution characters.</para> |
<emphasis>[]</emphasis>, which is the Tcl execution characters.</para> |
2407 |
|
|
2408 |
<example> |
<example> |
2409 |
<title>Example Board Config File</title> |
<title>Example Board Config File</title> |
2784 |
|
|
2785 |
<sect1 id=writing xreflabel="Writing A Test Case"> |
<sect1 id=writing xreflabel="Writing A Test Case"> |
2786 |
<title>Writing A Test Case</title> |
<title>Writing A Test Case</title> |
2787 |
|
|
2788 |
<para>The easiest way to prepare a new test case is to base it |
<para>The easiest way to prepare a new test case is to base it |
2789 |
on an existing one for a similar situation. There are two major |
on an existing one for a similar situation. There are two major |
2790 |
categories of tests: batch or interactive. Batch oriented tests |
categories of tests: batch or interactive. Batch oriented tests |
2842 |
|
|
2843 |
<sect1 id=debugging xreflabel="Debugging A Test Case"> |
<sect1 id=debugging xreflabel="Debugging A Test Case"> |
2844 |
<title>Debugging A Test Case</title> |
<title>Debugging A Test Case</title> |
2845 |
|
|
2846 |
<para>These are the kinds of debugging information available |
<para>These are the kinds of debugging information available |
2847 |
from DejaGnu:</para> |
from DejaGnu:</para> |
2848 |
|
|
2899 |
|
|
2900 |
<sect1 id=adding xreflabel="Adding A Test Case To A Testsuite"> |
<sect1 id=adding xreflabel="Adding A Test Case To A Testsuite"> |
2901 |
<title>Adding A Test Case To A Testsuite.</title> |
<title>Adding A Test Case To A Testsuite.</title> |
2902 |
|
|
2903 |
<para>There are two slightly different ways to add a test |
<para>There are two slightly different ways to add a test |
2904 |
case. One is to add the test case to an existing directory. The |
case. One is to add the test case to an existing directory. The |
2905 |
other is to create a new directory to hold your test. The |
other is to create a new directory to hold your test. The |
2948 |
<listitem><para>To add support in the new directory for |
<listitem><para>To add support in the new directory for |
2949 |
configure and make, you must also create a |
configure and make, you must also create a |
2950 |
<filename>Makefile.in</filename> and a |
<filename>Makefile.in</filename> and a |
2951 |
<filename>configure.in</filename>.</para></listitem> |
<filename>configure.in</filename>.</para></listitem> |
2952 |
</itemizedlist> |
</itemizedlist> |
2953 |
|
|
2954 |
</sect1> |
</sect1> |
2955 |
|
|
2956 |
<sect1 id=hints xreflabel="Hints On Writing A Test Case"> |
<sect1 id=hints xreflabel="Hints On Writing A Test Case"> |
2957 |
<title>Hints On Writing A Test Case</title> |
<title>Hints On Writing A Test Case</title> |
2958 |
|
|
2959 |
<para>It is safest to write patterns that match all the output |
<para>It is safest to write patterns that match all the output |
2960 |
generated by the tested program; this is called closure. |
generated by the tested program; this is called closure. |
2961 |
If a pattern does not match the entire output, any output that |
If a pattern does not match the entire output, any output that |
3014 |
|
|
3015 |
<sect1 id=tvariables xreflabel="Test Case Variables"> |
<sect1 id=tvariables xreflabel="Test Case Variables"> |
3016 |
<title>Special variables used by test cases.</title> |
<title>Special variables used by test cases.</title> |
3017 |
|
|
3018 |
<para>There are special variables used by test cases. These contain |
<para>There are special variables used by test cases. These contain |
3019 |
other information from DejaGnu. Your test cases can use these variables, |
other information from DejaGnu. Your test cases can use these variables, |
3020 |
with conventional meanings (as well as the variables saved in |
with conventional meanings (as well as the variables saved in |
3063 |
<function>${tool}_start</function> command. This is conventionally |
<function>${tool}_start</function> command. This is conventionally |
3064 |
used for batch oriented programs, like GCC and GAS, that may |
used for batch oriented programs, like GCC and GAS, that may |
3065 |
produce interesting output (warnings, errors) without further |
produce interesting output (warnings, errors) without further |
3066 |
interaction.</para></listitem> |
interaction.</para></listitem> |
3067 |
</varlistentry> |
</varlistentry> |
3068 |
</variablelist> |
</variablelist> |
3069 |
|
|
3070 |
</sect1> |
</sect1> |
3071 |
|
|
3072 |
</chapter> |
</chapter> |
3073 |
|
|
3074 |
<chapter id=unit xreflabel="Unit Testing"> |
<chapter id=unit xreflabel="Unit Testing"> |
3075 |
<title>Unit Testing</title> |
<title>Unit Testing</title> |
3076 |
|
|
3077 |
<sect1 id=unittest xreflabel="What Is Unit Testing ?"> |
<sect1 id=unittest xreflabel="What Is Unit Testing ?"> |
3078 |
<title>What Is Unit Testing ?</title> |
<title>What Is Unit Testing ?</title> |
3079 |
|
|
3080 |
<para>Most regression testing as done by DejaGnu is system |
<para>Most regression testing as done by DejaGnu is system |
3081 |
testing. This is the complete application is tested all at |
testing. This is the complete application is tested all at |
3082 |
once. Unit testing is for testing single files, or small |
once. Unit testing is for testing single files, or small |
3095 |
|
|
3096 |
<sect1 id=djh xreflabel="The dejagnu.h Header File"> |
<sect1 id=djh xreflabel="The dejagnu.h Header File"> |
3097 |
<title>The dejagnu.h Header File</title> |
<title>The dejagnu.h Header File</title> |
3098 |
|
|
3099 |
<para>DejaGnu uses a single header file to assist in unit |
<para>DejaGnu uses a single header file to assist in unit |
3100 |
testing. As this file also produces it's one test state output, |
testing. As this file also produces it's one test state output, |
3101 |
it can be run standalone, which is very useful for testing on |
it can be run standalone, which is very useful for testing on |
3107 |
messages, and then merge them into DejaGnu's.</para> |
messages, and then merge them into DejaGnu's.</para> |
3108 |
|
|
3109 |
</sect1> |
</sect1> |
3110 |
|
|
3111 |
</chapter> |
</chapter> |
3112 |
|
|
3113 |
<!-- Keep this comment at the end of the file |
<!-- Keep this comment at the end of the file |