diff options
| author | Misha Brukman <brukman+llvm@gmail.com> | 2004-03-01 18:21:04 +0000 |
|---|---|---|
| committer | Misha Brukman <brukman+llvm@gmail.com> | 2004-03-01 18:21:04 +0000 |
| commit | 1d83e111016bcfb3502545765151c7592acc844a (patch) | |
| tree | a7217978af8bb3fef533dd4a583e6179f298af8c /docs/TestingGuide.html | |
| parent | daa4cb0f03959cfc884e71eee3f83aef8138dba4 (diff) | |
| download | external_llvm-1d83e111016bcfb3502545765151c7592acc844a.zip external_llvm-1d83e111016bcfb3502545765151c7592acc844a.tar.gz external_llvm-1d83e111016bcfb3502545765151c7592acc844a.tar.bz2 | |
* HTML 4.01 Strict compliance
* Removed unnecessary tabs in the entire file
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@12051 91177308-0d34-0410-b5e6-96231b3b80d8
Diffstat (limited to 'docs/TestingGuide.html')
| -rw-r--r-- | docs/TestingGuide.html | 756 |
1 files changed, 355 insertions, 401 deletions
diff --git a/docs/TestingGuide.html b/docs/TestingGuide.html index 3c19cd1..bfe8316 100644 --- a/docs/TestingGuide.html +++ b/docs/TestingGuide.html @@ -1,11 +1,10 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" + "http://www.w3.org/TR/html4/strict.dtd"> <html> <head> - <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> - <link rel="stylesheet" href="llvm.css" type="text/css" media="screen" /> - <title>LLVM Test Suite Guide</title> + <title>LLVM Test Suite Guide</title> + <link rel="stylesheet" href="llvm.css" type="text/css"> </head> - <body> <div class="doc_title"> @@ -16,414 +15,369 @@ <li><a href="#overview">Overview</a></li> <li><a href="#Requirements">Requirements</a></li> <li><a href="#quick">Quick Start</a></li> -<li><a href="#org">LLVM Test Suite Organization</a></li> +<li><a href="#org">LLVM Test Suite Organization</a> <ul> <li><a href="#codefragments">Code Fragments</a></li> <li><a href="#wholeprograms">Whole Programs</a></li> -</ul> +</ul></li> <li><a href="#tree">LLVM Test Suite Tree</a></li> <li><a href="#qmstructure">QMTest Structure</a></li> <li><a href="#progstructure">Programs Structure</a></li> <li><a href="#run">Running the LLVM Tests</a></li> -<p><b>Written by John T. Criswell</b></p> </ol> - <!--===============================================================--> - <div class="doc_section"><a name="overview">Overview</a></div> - <!--===============================================================--> - - <div class="doc_text"> - <p> - This document is the reference manual for the LLVM test suite. It - documents the structure of the LLVM test suite, the tools needed to - use it, and how to add and run tests. - </p> - </div> - - <!--===============================================================--> - <div class="doc_section"><a name="Requirements">Requirements</a></div> - <!--===============================================================--> - - <div class="doc_text"> - <p> - In order to use the LLVM test suite, you will need all of the software - required to build LLVM, plus the following: - </p> - <dl compact> - <dt><A HREF="http://www.qmtest.com">QMTest</A></dt> - <dd>The LLVM test suite uses QMTest to organize and - run tests.</dd> - - <dt><A HREF="http://www.python.org">Python</A></dt> - <dd>You will need a Python interpreter that works with - QMTest. Python will need zlib and SAX support - enabled.</dd> - </dl> - </div> - - <!--===============================================================--> - <div class="doc_section"><a name="quick">Quick Start</a></div> - <!--===============================================================--> - - <div class="doc_text"> - <p> - The tests are located in the LLVM source tree under the directory - <tt>llvm/test</tt>. To run all of the tests in LLVM, use the Master - Makefile in that directory: - </p> - <pre> - % gmake -C llvm/test - </pre> - - <p> - To run only the code fragment tests (i.e. those that do basic testing of - LLVM), run the tests organized by QMTest: - </p> - - <pre> - % gmake -C llvm/test qmtest - </pre> - - <p> - To run only the tests that compile and execute whole programs, run the - Programs tests: - </p> - - <pre> - % gmake -C llvm/test/Programs - </pre> - </div> - - <!--===============================================================--> - <div class="doc_section"><h2><a name="org">LLVM Test Suite - Organization </a></h2></div> - <!--===============================================================--> - - <div class="doc_text"> - <p>The LLVM test suite contains two major categories of tests: code - fragments and whole programs.</p> - </div> - - <div class="doc_subsection"><a name="codefragments">Code Fragments</a> - </div> - - <div class="doc_text"> - <p> - Code fragments are small pieces of code that test a specific - feature of LLVM or trigger a specific bug in LLVM. They are - usually written in LLVM assembly language, but can be - written in other languages if the test targets a - particular language front end. - </p><p> - Code fragments are not complete programs, and they are - never executed to determine correct behavior. - </p><p> - The tests in the Features and - Regression directories contain code fragments. - </p> - </div> - - <div class="doc_subsection"><a name="wholeprograms">Whole Programs</a> - </div> - - <div class="doc_text"> - <p> - Whole Programs are pieces of code which can be compiled and - linked into a stand-alone program that can be executed. These - programs are generally written in high level languages such as C - or C++, but sometimes they are written straight in LLVM - assembly. - </p><p> - These programs are compiled and then executed using several - different methods (native compiler, LLVM C backend, LLVM JIT, - LLVM native code generation, etc). The output of these programs - is compared to ensure that LLVM is compiling the program - correctly. - </p><p> - In addition to compiling and executing programs, whole program - tests serve as a way of benchmarking LLVM performance, both in - terms of the efficiency of the programs generated as well as the - speed with which LLVM compiles, optimizes, and generates code. - </p><p> - The Programs directory contains all tests which compile and - benchmark whole programs. - </p> - </div> - - <!--===============================================================--> - <div class="doc_section"><h2><a name="tree">LLVM Test Suite Tree</a> - </div> - <!--===============================================================--> - - <div class="doc_text"> - <p>Each type of test in the LLVM test suite has its own directory. The - major subtrees of the test suite directory tree are as follows:</p> +<p><b>Written by John T. Criswell</b></p> + +<!--===============================================================--> +<div class="doc_section"><a name="overview">Overview</a></div> +<!--===============================================================--> + +<div class="doc_text"> + +<p>This document is the reference manual for the LLVM test suite. It documents +the structure of the LLVM test suite, the tools needed to use it, and how to add +and run tests.</p> + +</div> + +<!--===============================================================--> +<div class="doc_section"><a name="Requirements">Requirements</a></div> +<!--===============================================================--> + +<div class="doc_text"> + +<p>In order to use the LLVM test suite, you will need all of the software +required to build LLVM, plus the following:</p> + +<dl> + <dt><a href="http://www.qmtest.com">QMTest</A></dt> + <dd>The LLVM test suite uses QMTest to organize and run tests.</dd> + + <dt><a href="http://www.python.org">Python</A></dt> + <dd>You will need a Python interpreter that works with QMTest. Python will + need zlib and SAX support enabled.</dd> +</dl> + +</div> + +<!--===============================================================--> +<div class="doc_section"><a name="quick">Quick Start</a></div> +<!--===============================================================--> + +<div class="doc_text"> + +<p> The tests are located in the LLVM source tree under the directory +<tt>llvm/test</tt>. To run all of the tests in LLVM, use the Master Makefile in +that directory:</p> + +<pre> + % gmake -C llvm/test +</pre> + +<p>To run only the code fragment tests (i.e. those that do basic testing of +LLVM), run the tests organized by QMTest:</p> + +<pre> + % gmake -C llvm/test qmtest +</pre> + +<p>To run only the tests that compile and execute whole programs, run the +Programs tests:</p> + +<pre> + % gmake -C llvm/test/Programs +</pre> + +</div> + +<!--===============================================================--> +<div class="doc_section"><a name="org">LLVM Test Suite Organization</a></div> +<!--===============================================================--> + +<div class="doc_text"> + +<p>The LLVM test suite contains two major categories of tests: code +fragments and whole programs.</p> + +</div> + +<div class="doc_subsection"><a name="codefragments">Code Fragments</a> +</div> + +<div class="doc_text"> + +<p>Code fragments are small pieces of code that test a specific feature of LLVM +or trigger a specific bug in LLVM. They are usually written in LLVM assembly +language, but can be written in other languages if the test targets a particular +language front end.</p> + +<p>Code fragments are not complete programs, and they are never executed to +determine correct behavior.</p> + +<p>The tests in the Features and Regression directories contain code +fragments.</p> + +</div> + +<div class="doc_subsection"><a name="wholeprograms">Whole Programs</a></div> + +<div class="doc_text"> + +<p>Whole Programs are pieces of code which can be compiled and linked into a +stand-alone program that can be executed. These programs are generally written +in high level languages such as C or C++, but sometimes they are written +straight in LLVM assembly.</p> + +<p>These programs are compiled and then executed using several different +methods (native compiler, LLVM C backend, LLVM JIT, LLVM native code generation, +etc). The output of these programs is compared to ensure that LLVM is compiling +the program correctly.</p> + +<p>In addition to compiling and executing programs, whole program tests serve as +a way of benchmarking LLVM performance, both in terms of the efficiency of the +programs generated as well as the speed with which LLVM compiles, optimizes, and +generates code.</p> + +<p>The Programs directory contains all tests which compile and benchmark whole +programs.</p> + +</div> + +<!--===============================================================--> +<div class="doc_section"><a name="tree">LLVM Test Suite Tree</a></div> +<!--===============================================================--> + +<div class="doc_text"> + +<p>Each type of test in the LLVM test suite has its own directory. The major +subtrees of the test suite directory tree are as follows:</p> + +<ul> + <li>Features + <p>This directory contains sample codes that test various features of the + LLVM language. These pieces of sample code are run through various + assembler, disassembler, and optimizer passes.</p> + + <li>Regression + <p>This directory contains regression tests for LLVM. When a bug is found + in LLVM, a regression test containing just enough code to reproduce the + problem should be written and placed somewhere underneath this directory. + In most cases, this will be a small piece of LLVM assembly language code, + often distilled from an actual application or benchmark.</p> + + <li>Programs + <p>The Programs directory contains programs that can be compiled with LLVM + and executed. These programs are compiled using the native compiler and + various LLVM backends. The output from the program compiled with the native + compiler is assumed correct; the results from the other programs are + compared to the native program output and pass if they match. </p> + + <p> In addition for testing correctness, the Programs directory also + performs timing tests of various LLVM optimizations. It also records + compilation times for the compilers and the JIT. This information can be + used to compare the effectiveness of LLVM's optimizations and code + generation.</p> + + <p>The Programs directory is subdivided into several smaller subdirectories: + </p> + + <ul> + <li>Programs/SingleSource + <p>The SingleSource directory contains test programs that are only a + single source file in size. These are usually small benchmark programs + or small programs that calculate a particular value. Several such + programs are grouped together in each directory.</p></li> + + <li>Programs/MultiSource + <p>The MultiSource directory contains subdirectories which contain + entire programs with multiple source files. Large benchmarks and whole + applications go here.</p></li> + + <li>Programs/External + <p>The External directory contains Makefiles for building code that is + external to (i.e. not distributed with) LLVM. The most prominent member + of this directory is the SPEC 2000 benchmark suite. The presence and + location of these external programs is configured by the LLVM + <tt>configure</tt> script.</p></li> - <ul> - <li>Features - <p> - This directory contains sample codes that test various features - of the LLVM language. These pieces of sample code are run - through various assembler, disassembler, and optimizer passes. - </p> - - <li>Regression - <p> - This directory contains regression tests for LLVM. When a bug - is found in LLVM, a regression test containing just enough - code to reproduce the problem should be written and placed - somewhere underneath this directory. In most cases, this - will be a small piece of LLVM assembly language code, often - distilled from an actual application or benchmark. - </p> - - <li>Programs - <p> - The Programs directory contains programs that can be compiled - with LLVM and executed. These programs are compiled using the - native compiler and various LLVM backends. The output from the - program compiled with the native compiler is assumed correct; - the results from the other programs are compared to the native - program output and pass if they match. - </p><p> - In addition for testing correctness, the Programs directory - also performs timing tests of various LLVM optimizations. - It also records compilation times for the compilers and the - JIT. This information can be used to compare the - effectiveness of LLVM's optimizations and code generation. - </p><p> - The Programs directory is subdivided into several smaller - subdirectories: - </p> - - <ul> - <li>Programs/SingleSource - <p> - The SingleSource directory contains test programs that - are only a single source file in size. These are - usually small benchmark programs or small programs that - calculate a particular value. Several such programs are - grouped together in each directory. - </p> - - <li>Programs/MultiSource - <p> - The MultiSource directory contains subdirectories which - contain entire programs with multiple source files. - Large benchmarks and whole applications go here. - </p> - - <li>Programs/External - <p> - The External directory contains Makefiles for building - code that is external to (i.e. not distributed with) - LLVM. The most prominent member of this directory is - the SPEC 2000 benchmark suite. The presence and - location of these external programs is configured by the - LLVM <tt>configure</tt> script. - </p> - </ul> - - <p> - - <li>QMTest - <p> - This directory contains the QMTest information files. Inside - this directory are QMTest administration files and the Python - code that implements the LLVM test and database classes. - </p> - </ul> - </div> - - <!--===============================================================--> - <div class="doc_section"><h2><a name="qmstructure">QMTest Structure</a> - </div> - <!--===============================================================--> - - <div class="doc_text"> - <p> - The LLVM test suite is partially driven by QMTest and partially - driven by GNU Make. Specifically, the Features and Regression tests - are all driven by QMTest. The Programs directory is currently - driven by a set of Makefiles. - </p><p> - The QMTest system needs to have several pieces of information - available; these pieces of configuration information are known - collectively as the "context" in QMTest parlance. Since the context - for LLVM is relatively large, the master Makefile in llvm/test - sets it for you. - </p><p> - The LLVM database class makes the subdirectories of llvm/test a - QMTest test database. For each directory that contains tests driven by - QMTest, it knows what type of test the source file is and how to run it. - </p><p> - Hence, the QMTest namespace is essentially what you see in the - Feature and Regression directories, but there is some magic that - the database class performs (as described below). - </p><p> - The QMTest namespace is currently composed of the following tests and - test suites: - </p> - - <ul> - <li>Feature - <p> - These are the feature tests found in the Feature directory. - They are broken up into the following categories: - </p> - <ul> - <li>ad - <p> - Assembler/Disassembler tests. These tests verify that a - piece of LLVM assembly language can be assembled into - bytecode and then disassembled into the original - assembly language code. It does this several times to - ensure that assembled output can be disassembled and - disassembler output can be assembled. It also verifies - that the give assembly language file can be assembled - correctly. - </p> - - <li>opt - <p> - Optimizer tests. These tests verify that two of the - optimizer passes completely optimize a program (i.e. - after a single pass, they cannot optimize a program - any further). - </p> - - <li>mc - <p> - Machine code tests. These tests verify that the LLVM - assembly language file can be translated into native - assembly code. - </p> - - <li>cc - <p> - C code tests. These tests verify that the specified - LLVM assembly code can be converted into C source code - using the C backend. - </p> - </ul> - - <p> - The LLVM database class looks at every file in the Feature - directory and creates a fake test hierarchy containing - <tt>Feature.<testtype>.<testname></tt>. So, if you - add an LLVM assembly language file to the Feature directory, it - actually creates 5 new tests: assembler/disassembler, assembler, - optimizer, machine code, and C code. - </p> - - <li>Regression - <p> - These are the regression tests. There is one suite for each - subdirectory of the Regression directory. If you add a new - subdirectory there, you will need to modify, at least, the - <tt>RegressionMap</tt> variable in <tt>QMTest/llvmdb.py</tt> so - that QMTest knows how to run the tests in the new subdirectory. - </p> - </ul> - </div> - - <!--===============================================================--> - <div class="doc_section"><h2><a name="progstructure">Programs - Structure</a></div> - <!--===============================================================--> - - <div class="doc_text"> - <p> - As mentioned previously, the Programs tree in llvm/test provides three - types of tests: MultiSource, SingleSource, and External. Each tree is - then subdivided into several categories, including applications, - benchmarks, regression tests, code that is strange grammatically, etc. - These organizations should be relatively self explanatory. - </p><p> - In addition to the regular Programs tests, the Programs tree also - provides a mechanism for compiling the programs in different ways. If - the variable TEST is defined on the gmake command line, the test system - will include a Makefile named <tt>TEST.<value of TEST - variable>.Makefile</tt>. This Makefile can modify build rules to - yield different results. - </p><p> - For example, the LLVM nightly tester uses <tt>TEST.nightly.Makefile</tt> - to create the nightly test reports. To run the nightly tests, run - <tt>gmake TEST=nightly</tt>. - </p><p> - There are several TEST Makefiles available in the tree. Some of them - are designed for internal LLVM research and will not work outside of the - LLVM research group. They may still be valuable, however, as a guide to - writing your own TEST Makefile for any optimization or analysis passes - that you develop with LLVM. - </p> - </div> - - <!--===============================================================--> - <div class="doc_section"><h2><a name="run">Running the LLVM Tests</a> - </div> - <!--===============================================================--> - - <div class="doc_text"> - <p> - First, all tests are executed within the LLVM object directory tree. - They <i>are not</i> executed inside of the LLVM source tree. This is - because the test suite creates temporary files during execution. - </p><p> - The master Makefile in llvm/test is capable of running both the - QMTest driven tests and the Programs tests. By default, it will run - all of the tests. - </p><p> - To run only the QMTest driven tests, run <tt>gmake qmtest</tt> at the - command line in llvm/tests. To run a specific qmtest, suffix the test - name with ".t" when running gmake. - </p><p> - For example, to run the Regression.LLC tests, type - <tt>gmake Regression.LLC.t</tt> in llvm/tests. - </p><p> - Note that the Makefiles in llvm/test/Features and llvm/test/Regression - are gone. You must now use QMTest from the llvm/test directory to run - them. - </p><p> - To run the Programs test, cd into the llvm/test/Programs directory and - type <tt>gmake</tt>. Alternatively, you can type <tt>gmake - TEST=<type> test</tt> to run one of the specialized tests in - llvm/test/Programs/TEST.<type>.Makefile. For example, you could - run the nightly tester tests using the following commands: - </p> - - <pre> - % cd llvm/test/Programs - % gmake TEST=nightly test - </pre> - - <p> - Regardless of which test you're running, the results are printed on - standard output and standard error. You can redirect these results to a - file if you choose. - </p><p> - Some tests are known to fail. Some are bugs that we have not fixed yet; - others are features that we haven't added yet (or may never add). In - QMTest, the result for such tests will be XFAIL (eXpected FAILure). In - this way, you can tell the difference between an expected and unexpected - failure. - </p><p> - The Programs tests have no such feature as of this time. If the test - passes, only warnings and other miscellaneous output will be generated. - If a test fails, a large <program> FAILED message will be - displayed. This will help you separate benign warnings from actual test - failures. - </p> - </div> + </ul></li> -<!-- *********************************************************************** --> + <li>QMTest + <p>This directory contains the QMTest information files. Inside this + directory are QMTest administration files and the Python code that + implements the LLVM test and database classes.</p> + +</ul> + +</div> + +<!--===============================================================--> +<div class="doc_section"><a name="qmstructure">QMTest Structure</a></div> +<!--===============================================================--> + +<div class="doc_text"> + +<p>The LLVM test suite is partially driven by QMTest and partially +driven by GNU Make. Specifically, the Features and Regression tests +are all driven by QMTest. The Programs directory is currently +driven by a set of Makefiles.</p> + +<p>The QMTest system needs to have several pieces of information +available; these pieces of configuration information are known +collectively as the "context" in QMTest parlance. Since the context +for LLVM is relatively large, the master Makefile in llvm/test +sets it for you.</p> + +<p>The LLVM database class makes the subdirectories of llvm/test a +QMTest test database. For each directory that contains tests driven by +QMTest, it knows what type of test the source file is and how to run it.</p> -<hr><font size="-1"> -<address>John T. Criswell</address> -<a href="http://llvm.cs.uiuc.edu">The LLVM Compiler Infrastructure</a> -<br> -Last modified: $Date$ -</font> +<p>Hence, the QMTest namespace is essentially what you see in the +Feature and Regression directories, but there is some magic that +the database class performs (as described below).</p> + +<p>The QMTest namespace is currently composed of the following tests and test +suites:</p> + +<ul> + <li>Feature + <p> + These are the feature tests found in the Feature directory. + They are broken up into the following categories: + </p> + <ul> + <li>ad + <p>Assembler/Disassembler tests. These tests verify that a piece of LLVM + assembly language can be assembled into bytecode and then disassembled + into the original assembly language code. It does this several times to + ensure that assembled output can be disassembled and disassembler output + can be assembled. It also verifies that the give assembly language file + can be assembled correctly.</p></li> + + <li>opt + <p>Optimizer tests. These tests verify that two of the optimizer passes + completely optimize a program (i.e. after a single pass, they cannot + optimize a program any further).</p></li> + + <li>mc + <p> Machine code tests. These tests verify that the LLVM assembly + language file can be translated into native assembly code.</p></li> + + <li>cc + <p>C code tests. These tests verify that the specified LLVM assembly + code can be converted into C source code using the C backend.</p></li> + </ul> + + <p>The LLVM database class looks at every file in the Feature directory and + creates a fake test hierarchy containing + <tt>Feature.<testtype>.<testname></tt>. So, if you add an LLVM + assembly language file to the Feature directory, it actually creates 5 new + tests: assembler/disassembler, assembler, optimizer, machine code, and C code. + </p> + + <li>Regression + <p>These are the regression tests. There is one suite for each + subdirectory of the Regression directory. If you add a new subdirectory + there, you will need to modify, at least, the <tt>RegressionMap</tt> + variable in <tt>QMTest/llvmdb.py</tt> so that QMTest knows how to run the + tests in the new subdirectory.</p> + +</ul> + +</div> + +<!--===============================================================--> +<div class="doc_section"><a name="progstructure">Programs Structure</a></div> +<!--===============================================================--> + +<div class="doc_text"> + +<p>As mentioned previously, the Programs tree in llvm/test provides three types +of tests: MultiSource, SingleSource, and External. Each tree is then subdivided +into several categories, including applications, benchmarks, regression tests, +code that is strange grammatically, etc. These organizations should be +relatively self explanatory.</p> + +<p>In addition to the regular Programs tests, the Programs tree also provides a +mechanism for compiling the programs in different ways. If the variable TEST is +defined on the gmake command line, the test system will include a Makefile named +<tt>TEST.<value of TEST variable>.Makefile</tt>. This Makefile can modify +build rules to yield different results.</p> + +<p>For example, the LLVM nightly tester uses <tt>TEST.nightly.Makefile</tt> to +create the nightly test reports. To run the nightly tests, run <tt>gmake +TEST=nightly</tt>.</p> + +<p>There are several TEST Makefiles available in the tree. Some of them are +designed for internal LLVM research and will not work outside of the LLVM +research group. They may still be valuable, however, as a guide to writing your +own TEST Makefile for any optimization or analysis passes that you develop with +LLVM.</p> + +</div> + +<!--===============================================================--> +<div class="doc_section"><a name="run">Running the LLVM Tests</a></div> +<!--===============================================================--> + +<div class="doc_text"> + +<p>First, all tests are executed within the LLVM object directory tree. They +<i>are not</i> executed inside of the LLVM source tree. This is because the +test suite creates temporary files during execution. </p> + +<p>The master Makefile in llvm/test is capable of running both the QMTest driven +tests and the Programs tests. By default, it will run all of the tests.</p> + +<p>To run only the QMTest driven tests, run <tt>gmake qmtest</tt> at the +command line in llvm/tests. To run a specific qmtest, suffix the test name with +".t" when running gmake.</p> + +<p>For example, to run the Regression.LLC tests, type <tt>gmake +Regression.LLC.t</tt> in llvm/tests.</p> + +<p>Note that the Makefiles in llvm/test/Features and llvm/test/Regression are +gone. You must now use QMTest from the llvm/test directory to run them.</p> + +<p>To run the Programs test, cd into the llvm/test/Programs directory and type +<tt>gmake</tt>. Alternatively, you can type <tt>gmake TEST=<type> +test</tt> to run one of the specialized tests in +llvm/test/Programs/TEST.<type>.Makefile. For example, you could run the +nightly tester tests using the following commands:</p> + +<pre> + % cd llvm/test/Programs + % gmake TEST=nightly test +</pre> + +<p>Regardless of which test you're running, the results are printed on standard +output and standard error. You can redirect these results to a file if you +choose.</p> + +<p>Some tests are known to fail. Some are bugs that we have not fixed yet; +others are features that we haven't added yet (or may never add). In QMTest, +the result for such tests will be XFAIL (eXpected FAILure). In this way, you +can tell the difference between an expected and unexpected failure.</p> + +<p>The Programs tests have no such feature as of this time. If the test passes, +only warnings and other miscellaneous output will be generated. If a test +fails, a large <program> FAILED message will be displayed. This will help +you separate benign warnings from actual test failures.</p> + +</div> + +<!-- *********************************************************************** --> +<hr> +<address> + <a href="http://jigsaw.w3.org/css-validator/check/referer"><img + src="http://jigsaw.w3.org/css-validator/images/vcss" alt="Valid CSS!"></a> + <a href="http://validator.w3.org/check/referer"><img + src="http://www.w3.org/Icons/valid-html401" alt="Valid HTML 4.01!" /></a> + + John T. Criswell<br> + <a href="http://llvm.cs.uiuc.edu">The LLVM Compiler Infrastructure</a><br> + Last modified: $Date$ +</address> </body> </html> |