/[dejagnu]/dejagnu/doc/user.sgml
ViewVC logotype

Diff of /dejagnu/doc/user.sgml

Parent Directory Parent Directory | Revision Log Revision Log | View Patch Patch

revision 1.11 by bje, Tue Jul 29 11:57:09 2003 UTC revision 1.12 by bje, Sat Aug 16 06:31:50 2003 UTC
# Line 711  powerpc-linux-gcc -g -O2 -o calc calc.o Line 711  powerpc-linux-gcc -g -O2 -o calc calc.o
711   <chapter id=runningtests>   <chapter id=runningtests>
712      <title>Running Tests</title>      <title>Running Tests</title>
713    
714      <para>There are two ways to execute a test suite. The most      <para>There are two ways to execute a testsuite. The most
715      common way is when there is existing support in the      common way is when there is existing support in the
716      <filename>Makefile</filename>. This support consists of a      <filename>Makefile</filename>. This support consists of a
717      <emphasis>check</emphasis> target. The other way is to execute the      <emphasis>check</emphasis> target. The other way is to execute the
# Line 812  powerpc-linux-gcc -g -O2 -o calc calc.o Line 812  powerpc-linux-gcc -g -O2 -o calc calc.o
812          <varlistentry>          <varlistentry>
813            <term>UNRESOLVED</term>            <term>UNRESOLVED</term>
814            <listitem><para>Output from a test requires manual inspection; the            <listitem><para>Output from a test requires manual inspection; the
815            test suite could not automatically determine the outcome.  For            testsuite could not automatically determine the outcome.  For
816            example, your tests can report this outcome is when a test does not            example, your tests can report this outcome is when a test does not
817            complete as expected.</para></listitem>            complete as expected.</para></listitem>
818          </varlistentry>          </varlistentry>
# Line 844  powerpc-linux-gcc -g -O2 -o calc calc.o Line 844  powerpc-linux-gcc -g -O2 -o calc calc.o
844            <listitem><para>Indicates a major problem (detected by the test case            <listitem><para>Indicates a major problem (detected by the test case
845            itself) in running the test. This is usually an unrecoverable error,            itself) in running the test. This is usually an unrecoverable error,
846            such as a missing file or loss of communication to the target. (POSIX            such as a missing file or loss of communication to the target. (POSIX
847            test suites should not emit this message; use            testsuites should not emit this message; use
848            <emphasis>UNSUPPORTED</emphasis>, <emphasis>UNTESTED</emphasis>, or            <emphasis>UNSUPPORTED</emphasis>, <emphasis>UNTESTED</emphasis>, or
849            <emphasis>UNRESOLVED</emphasis> instead, as            <emphasis>UNRESOLVED</emphasis> instead, as
850            appropriate.)</para></listitem>            appropriate.)</para></listitem>
# Line 1054  powerpc-linux-gcc -g -O2 -o calc calc.o Line 1054  powerpc-linux-gcc -g -O2 -o calc calc.o
1054                    
1055          <varlistentry id=tool-opt>          <varlistentry id=tool-opt>
1056            <term><option>--tool[name(s)]</option></term>            <term><option>--tool[name(s)]</option></term>
1057            <listitem><para>Specifies which test suite to run, and what            <listitem><para>Specifies which testsuite to run, and what
1058            initialization module to use. <option>--tool</option> is used            initialization module to use. <option>--tool</option> is used
1059            <emphasis>only</emphasis> for these two purposes. It is            <emphasis>only</emphasis> for these two purposes. It is
1060            <emphasis>not</emphasis> used to name the executable program to            <emphasis>not</emphasis> used to name the executable program to
# Line 1163  powerpc-linux-gcc -g -O2 -o calc calc.o Line 1163  powerpc-linux-gcc -g -O2 -o calc calc.o
1163                    
1164          <para>Typically, you don't need must to use any command-line options.          <para>Typically, you don't need must to use any command-line options.
1165          <option>--tool</option> used is only required when there are more than          <option>--tool</option> used is only required when there are more than
1166          one test suite in the same directory. The default options are in the          one testsuite in the same directory. The default options are in the
1167          local site.exp file, created by "make site.exp".</para>          local site.exp file, created by "make site.exp".</para>
1168    
1169          <para>For example, if the directory <filename>gdb/testsuite</filename>          <para>For example, if the directory <filename>gdb/testsuite</filename>
# Line 1465  powerpc-linux-gcc -g -O2 -o calc calc.o Line 1465  powerpc-linux-gcc -g -O2 -o calc calc.o
1465      DejaGnu test environment using Tcl variables.  This ties the      DejaGnu test environment using Tcl variables.  This ties the
1466      DejaGnu test scripts into the <command>configure</command> and      DejaGnu test scripts into the <command>configure</command> and
1467      <command>make</command> programs. If this file is setup correctly,      <command>make</command> programs. If this file is setup correctly,
1468      it is possible to execute a test suite merely by typing      it is possible to execute a testsuite merely by typing
1469      <command>runtest</command>.</para>      <command>runtest</command>.</para>
1470    
1471      <para>DejaGnu supports two <filename>site.exp</filename>      <para>DejaGnu supports two <filename>site.exp</filename>
# Line 1487  powerpc-linux-gcc -g -O2 -o calc calc.o Line 1487  powerpc-linux-gcc -g -O2 -o calc calc.o
1487      <symbol>DEJAGNU</symbol> to the name of the file. This is also      <symbol>DEJAGNU</symbol> to the name of the file. This is also
1488      refered to as the ``global'' config file.</para>      refered to as the ``global'' config file.</para>
1489    
1490      <para>Any directory containing a configured test suite also has a      <para>Any directory containing a configured testsuite also has a
1491      local <filename>site.exp</filename>, capturing configuration values      local <filename>site.exp</filename>, capturing configuration values
1492      specific to the tool under test.  Since <command>runtest</command>      specific to the tool under test.  Since <command>runtest</command>
1493      loads these values last, the individual test configuration can      loads these values last, the individual test configuration can
# Line 1496  powerpc-linux-gcc -g -O2 -o calc calc.o Line 1496  powerpc-linux-gcc -g -O2 -o calc calc.o
1496    
1497      <para>You can usually generate or update the testsuite's local      <para>You can usually generate or update the testsuite's local
1498      <filename>site.exp</filename> by typing <command>make      <filename>site.exp</filename> by typing <command>make
1499      site.exp</command> in the test suite directory, after the test      site.exp</command> in the testsuite directory, after the test
1500      suite is configured.</para>      suite is configured.</para>
1501    
1502      <para>You can also have a file in your home directory called      <para>You can also have a file in your home directory called
# Line 1519  powerpc-linux-gcc -g -O2 -o calc calc.o Line 1519  powerpc-linux-gcc -g -O2 -o calc calc.o
1519        local to each test directory, rather than in the global        local to each test directory, rather than in the global
1520        <filename>site.exp</filename> in the installed DejaGnu        <filename>site.exp</filename> in the installed DejaGnu
1521        library. This file is mostly for supplying tool specific info        library. This file is mostly for supplying tool specific info
1522        that is required by the test suite.</para>        that is required by the testsuite.</para>
1523    
1524        <para>All local <filename>site.exp</filename> files have        <para>All local <filename>site.exp</filename> files have
1525        two sections, separated by comment text. The first section is        two sections, separated by comment text. The first section is
# Line 1601  powerpc-linux-gcc -g -O2 -o calc calc.o Line 1601  powerpc-linux-gcc -g -O2 -o calc calc.o
1601      <para>This file defines the required fields for a local config      <para>This file defines the required fields for a local config
1602      file, namely the three config triplets, and the srcdir. It also      file, namely the three config triplets, and the srcdir. It also
1603      defines several other Tcl variables that are used exclusivly by      defines several other Tcl variables that are used exclusivly by
1604      the GCC test suite. For most test cases, the CXXFLAGS and LDFLAGS      the GCC testsuite. For most test cases, the CXXFLAGS and LDFLAGS
1605      are supplied by DejaGnu itself for cross testing, but to test a      are supplied by DejaGnu itself for cross testing, but to test a
1606      compiler, GCC needs to manipulate these itself.</para>      compiler, GCC needs to manipulate these itself.</para>
1607    
# Line 2067  powerpc-linux-gcc -g -O2 -o calc calc.o Line 2067  powerpc-linux-gcc -g -O2 -o calc calc.o
2067    <chapter id=Extending xreflabel="Extending DejaGnu">    <chapter id=Extending xreflabel="Extending DejaGnu">
2068      <title>Extending DejaGnu</title>      <title>Extending DejaGnu</title>
2069            
2070      <sect1 id=addsuite  xreflabel="Adding a new Test Suite">      <sect1 id=addsuite  xreflabel="Adding a new Testsuite">
2071        <title>Adding A New Test Suite</title>        <title>Adding A New Testsuite</title>
2072            
2073        <para>The testsuite for a new tool should always be located in that tools        <para>The testsuite for a new tool should always be located in that tools
2074        source directory. DejaGnu require the directory be named        source directory. DejaGnu require the directory be named
# Line 2084  powerpc-linux-gcc -g -O2 -o calc calc.o Line 2084  powerpc-linux-gcc -g -O2 -o calc calc.o
2084            
2085        <para>In general, the best way to learn how to write (code or even prose)        <para>In general, the best way to learn how to write (code or even prose)
2086        is to read something similar.  This principle applies to test cases and        is to read something similar.  This principle applies to test cases and
2087        to test suites.  Unfortunately, well-established test suites have a way        to testsuites.  Unfortunately, well-established testsuites have a way
2088        of developing their own conventions: as test writers become more        of developing their own conventions: as test writers become more
2089        experienced with DejaGnu and with Tcl, they accumulate more utilities,        experienced with DejaGnu and with Tcl, they accumulate more utilities,
2090        and take advantage of more and more features of        and take advantage of more and more features of
2091        <productname>Expect</productname> and <productname>Tcl</productname> in        <productname>Expect</productname> and <productname>Tcl</productname> in
2092        general.</para>        general.</para>
2093    
2094        <para>Inspecting such established test suites may make the prospect of        <para>Inspecting such established testsuites may make the prospect of
2095        creating an entirely new test suite appear overwhelming.  Nevertheless,        creating an entirely new testsuite appear overwhelming.  Nevertheless,
2096        it is quite straightforward to get a new test suite going.</para>        it is quite straightforward to get a new testsuite going.</para>
2097    
2098        <para>There is one test suite that is guaranteed not to grow more        <para>There is one testsuite that is guaranteed not to grow more
2099        elaborate over time: both it and the tool it tests were created expressly        elaborate over time: both it and the tool it tests were created expressly
2100        to illustrate what it takes to get started with DejaGnu.  The        to illustrate what it takes to get started with DejaGnu.  The
2101        <filename>example/</filename> directory of the DejaGnu distribution        <filename>example/</filename> directory of the DejaGnu distribution
2102        contains both an interactive tool called <command>calc</command>, and a        contains both an interactive tool called <command>calc</command>, and a
2103        test suite for it.  Reading this test suite, and experimenting with it,        testsuite for it.  Reading this testsuite, and experimenting with it,
2104        is a good way to supplement the information in this section.  (Thanks to        is a good way to supplement the information in this section.  (Thanks to
2105        Robert Lupton for creating calc and its test suite---and also the first        Robert Lupton for creating calc and its testsuite---and also the first
2106        version of this section of the manual!)</para>        version of this section of the manual!)</para>
2107    
2108        <para>To help orient you further in this task, here is an outline of the        <para>To help orient you further in this task, here is an outline of the
2109        steps to begin building a test suite for a program example.</para>        steps to begin building a testsuite for a program example.</para>
2110    
2111        <itemizedlist mark=bullet>        <itemizedlist mark=bullet>
2112                
# Line 2897  powerpc-linux-gcc -g -O2 -o calc calc.o Line 2897  powerpc-linux-gcc -g -O2 -o calc calc.o
2897    
2898      </sect1>      </sect1>
2899    
2900      <sect1 id=adding xreflabel="Adding A Test Case To A Test Suite">      <sect1 id=adding xreflabel="Adding A Test Case To A Testsuite">
2901        <title>Adding A Test Case To A Test Suite.</title>        <title>Adding A Test Case To A Testsuite.</title>
2902            
2903        <para>There are two slightly different ways to add a test        <para>There are two slightly different ways to add a test
2904        case. One is to add the test case to an existing directory. The        case. One is to add the test case to an existing directory. The
# Line 2937  powerpc-linux-gcc -g -O2 -o calc calc.o Line 2937  powerpc-linux-gcc -g -O2 -o calc calc.o
2937    
2938        <listitem><para>Add the new directory name to the        <listitem><para>Add the new directory name to the
2939        <symbol>configdirs</symbol> definition in the        <symbol>configdirs</symbol> definition in the
2940        <filename>configure.in</filename> file for the test suite        <filename>configure.in</filename> file for the testsuite
2941        directory. This way when <command>make</command> and        directory. This way when <command>make</command> and
2942        <command>configure</command> next run, they include the new        <command>configure</command> next run, they include the new
2943        directory.</para></listitem>        directory.</para></listitem>

Legend:
Removed from v.1.11  
changed lines
  Added in v.1.12

savannah-hackers-public@gnu.org
ViewVC Help
Powered by ViewVC 1.1.26