Xlib and X Protocol Test Suite
X Version 11 Release 6.1
User Guide for the X Test Suite
July 1992
Copyright © 1991, 1992
UniSoft Group Limited
Permission to use, copy, modify, distribute, and sell this
software and its documentation for any purpose is hereby
granted without fee, provided that the above copyright
notice appear in all copies and that both that copyright
notice and this permission notice appear in supporting
documentation, and that the name of UniSoft not be used in
advertising or publicity pertaining to distribution of the
software without specific, written prior permission. UniSoft
makes no representations about the suitability of this
software for any purpose. It is provided "as is"
without express or implied
warranty.
User Guide for the X Test Suite
User Guide for the X Test Suite
1. Introduction
This document is a guide to installing and running X Version
11 Release 6.1 of the X Test Suite. In order to do this,
please work through all of the steps described in this guide
in order. Information on the content, purpose and goals of
the X Test Suite is found in a series of appendices, which
follow the installation instructions.
Please read the "Release Notes for the X Test
Suite", which describe particular features of this
release.
Further information, which would be required by a programmer
to modify or extend the X Test Suite, is contained in a
separate document, "Programmers Guide for the X Test
Suite".
Included in this release is a version of the "Test
Environment Toolkit" ( TET ). This is
required to build and execute the X Test Suite. The
"Test Environment Toolkit" is a software tool
developed by X/Open, UNIX International, and the Open
Software Foundation. More details of the TET
appear in appendix E.
The contents of this document cover the installation and use
of the included version of the TET
.
- 1 -
User Guide for the X Test Suite
2. Preparation
This section of the User Guide describes how to check that
the system on which you want to build the X Test Suite has
the required utilities and sufficient disc space, how
to
check the version of the X Window System1 you wish to test,and how to extract the software from the supplieddistribution media. 2.1 Utilities The X Test Suite assumes that the following utilities areavailable on your system. 2.1.1 Bourne shell The configuration and building stages include exampleinstructions which have only been tested using the Bourneshell. The build configuration file sets the SHELL variable so thatthe Bourne shell will be used by make. No other settings forthis variable have been tested. 2.1.2 make The building stages assume the existence of make. 2.1.3 awk The report writer rpt uses awk. 2.1.4 Compiler A C compiler and link editor are required. The X Test Suiteassumes that when these utilities execute successfully, theyreturn a value of zero. The names of these utilities may beset in build configuration parameters. 2.1.5 Library archiver A library archiver and a means of ordering the libraries arerequired. The ordering software may be part of the libraryarchiver, the ranlib utility, or the utilities lorder andtsort. The names of these utilities may be set in buildconfiguration parameters. 2.1.6 File utilities The X Test Suite uses utilities to copy, move, remove andlink files during the build stages. The names of theseutilities may be set in build configuration parameters.
____________________
1.
The X Window System is a trademark of the Massachusetts
Institute of Technology.
X Window System Version 11 Release 4 is abbreviated to
X11R4 in this document.
X Window System Version 11 Release 5 is abbreviated to
X11R5 in this document.
- 2 -
User Guide for the X Test Suite
2.2 Checking your version of the X Window
System
If your version of the X Window System supports the
XTEST extension, you will be able to perform
tests for some assertions which are otherwise untestable.
The XTEST extension has been produced by MIT
since the initial release
of X11R5, based on a specification2 produced by UniSoft.The extension provides access to the X server to enabletesting of the following areas of the X Window System: —Those which rely on the simulation of device events. —Those requiring access to opaque client side datastructures. —Those requiring information on the cursor attribute ofwindows. Before you configure the X Test Suite, you should determinewhether your version of the X Window System includes theXTEST extension, and, if so, whether you wish to configureand build the X Test Suite to enable these features to betested. There are two things to check: 1. Check whether your X server supports the XTESTextension. This can be done by printing the list ofextensions available in your X server using the Xutility xdpyinfo. Note - the name of the extensionshould be printed exactly as in this User Guide -there are other testing extensions for X which are notcompatible with the XTEST extension. 2. Check whether you have the required libraries to linkthe test suite clients so as to access the XTESTextension. All test suite clients must be linked withXlib, which is normally named libX11.a. If you wantto access the XTEST extension, you will need twofurther libraries. These are the XTEST library(normally named libXtst.a) and the X extension library(normally named libXext.a). 2.3 Installing the X Test Suite Change to the directory in which you wish to install thedistribution. Set an environment variable TET_ROOT to thefull path name of that directory. Load the software from the media supplied into thatdirectory. The precise commands you should use depend onthe format of the media supplied to you, the utilitiesavailable on your system, the options supported by theutilities, and the names of the tape devices on your system.See the Release Notes for more information aboutinstallation.
____________________
2.
Drake, K.J., “Some Proposals for a Minimal X11 Testing
Extension.” UniSoft Ltd. June 1991
- 3 -
User Guide for the X Test Suite
3. Configuring the X Test Suite
This section contains instructions on all the procedures you
should go through in order to configure the X Test Suite,
before attempting to build it.
There is a description of the TET build tool
in section 3.1, and the relationship between the
TET build scheme and the Imake scheme in
section 3.2.
Sections 3.3 and 3.4 contain details of build and clean
configuration parameters, which you should edit to reflect
the configuration of the target platform on which the X Test
Suite is to be built.
Section 3.5 contains details of source files and include
files which contain system dependent data which cannot be
specified via the build configuration parameters. You should
check these files before configuration and if necessary edit
them to be suitable for your system.
3.1 The TET build tool
The TET provides a scheme to execute a build
tool, which builds the tests in the X Test Suite. The
execution of the build tool in the TET is
controlled by a small number of TET
configuration parameters, contained in a build configuration
file. These are described in section 3.3.1.
A build tool has been developed and is provided as part of
the X Test Suite. This is a shell script named
pmake, which is supplied in the directory
$TET_ROOT/xtest/bin. The shell script
pmake is an interface to the make system
command, and when invoked from the TET it
builds a test using the rules provided in a
Makefile.
Each Makefile in the X Test Suite is written
portably, using symbolic names to describe commands and
parameters which may vary from system to system. The values
of these symbolic names are all obtained by pmake
from additional parameters in the build configuration file
which are described in sections 3.3.2-3.3.7.
The pmake utility may be invoked directly from the
shell, as well as via the TET , to build
individual parts of the X Test Suite. This is described
further in subsequent sections of this guide.
There is also a clean tool pclean which is an
interface to the system make clean system command.
This uses parameters in a clean configuration
file.
3.2 Relationship between TET build scheme and
Imake
The TET is designed to provide a simple and
self contained interface to configure and build tests. The X
Test Suite can be configured and built with no specialised
knowledge of the X Window System beyond that contained in
the X Test Suite documentation, and using a limited set of
commonly available system commands. The only information
required to configure and build the X Test Suite is the
location of the X Window System Xlib and include files.
The X Window System itself includes a configuration scheme
which is known as Imake. This uses a utility
imake supplied as part of the X Window System to
create Makefiles from
- 4 -
User Guide for the X Test Suite
portable description files called
Imakefiles.
If you are familiar with the Imake scheme, and have
used it to configure and build the X Window System on the
platform being used to build the X Test Suite, you may be
able to set a limited number of the TET build
configuration variables described in section 3.3 to the same
value you used for an Imake variable. Where this is
possible, the name of the corresponding Imake
variable is cross referenced.
3.3 Build configuration parameters
All build configuration parameters are contained in a
configuration file that forms part of the TET
. This file should be edited to reflect the configuration of
the target machine. The file
$TET_ROOT/xtest/tetbuild.cfg
contains all the parameters that are needed to build theX Test Suite. The parameters are grouped in seven sectionswithin the configuration file. 3.3.1 Configuration Parameters defined by the TET None of these parameters require changing. They are alreadyset to defaults which are correct for the X Test Suite. TET_BUILD_TOOL The name of the program that the TET willexecute in build mode. You should use the pmake command that issupplied in the directory$TET_ROOT/xtest/bin.
Eg: TET_BUILD_TOOL=pmake
TET_BUILD_FILE Any flags required by the build tool. Thisparameter should be empty.
Eg: TET_BUILD_FILE=
TET_CLEAN_TOOL The name of the program that the TET willexecute in clean mode. You should use the pclean command that issupplied in the directory$TET_ROOT/xtest/bin.
Eg: TET_CLEAN_TOOL=pclean
- 5 -
User Guide for the X Test Suite
TET_CLEAN_FILE
Any flags required by the clean tool. This parameter should
be empty.
Eg: TET_CLEAN_FILE=
TET_OUTPUT_CAPTURE This flag is used by the TET to enable theoutput from the build tool to be saved andcopied into the journal file. This lineshould not be altered. 3.3.2 Configuration for system commands In this section the names of system commands are specified. SHELL The following line should cause the Bourneshell to be used by make.
Eg: SHELL=/bin/sh
CC A command to invoke the C compiler.
Eg: CC=cc
Imake variable: CcCmd RM A command to remove a file withoutinteractive help.
Eg: RM=rm -f
Imake variable: RmCmd AR A command to generate a library archive.
Eg: AR=ar crv
Imake variable: ArCmd LD A command to link object files.
Eg: LD=ld
Imake variable: LdCmd LN A command to make hard links.
Eg: LN=ln
- 6 -
User Guide for the X Test Suite
NB: This does not correspond to the Imake
variable: LnCmd
RANLIB If the system supports a command to order
library archives into random access libraries, then set the
parameter to that command. Otherwise it should be set to
echo (or a command that does nothing).
Eg: RANLIB=ranlib
Imake variable: RanlibCmd TSORT Set to cat if AR was set to a command whichinserts a symbol table in the libraryarchive, or if RANLIB was set to a commandwhich creates a random access library,otherwise set to tsort. LORDER Set to echo if AR was set to a command whichinserts a symbol table in the libraryarchive, or if RANLIB was set to a commandwhich creates a random access library.otherwise set to lorder. CP A command to copy files.
Eg: CP=cp
Imake variable: CpCmd CODEMAKER A utility to produce C source files fromdot-m files. The supplied utility mc shouldalways be used. This line should not bealtered.
Eg: CODEMAKER=mc
3.3.3 Configuration for the TET This section contains the locations of various parts of theTET. Usually only the first four parameters will needchanging, unless files have been moved from their defaultlocations. TET_ROOT The directory that contains all the files inthe X Test Suite. This should be set to thepath to which TET_ROOT was set (see thesection entitled "Installing the X TestSuite"). It must be written out as a fullpath without using any variable notation. TETBASE The directory that contains all the files inthe TET system. This is used for conveniencein defining the other directories. Thisshould be set to ${TET_ROOT}/tet.
- 7 -
User Guide for the X Test Suite
PORTINC An option that can be
given to the C compiler that will cause it to search all
directories that are required to allow portability to
systems that do not support POSIX . Should be
empty for POSIX systems. If compiling on a
BSD system using the supplied compatibility
library, then the following line should be used. (See
section entitled "The portability
library").
Eg: PORTINC=-I${TET_ROOT}/port/INC
PORTLIB A library containing POSIX.1 and C libraryfunctions that are not supplied by thesystem. This should be empty for a POSIXsystem. If compiling on a BSD system usingthe supplied compatibility library, then thefollowing line should be used. (See sectionentitled "The portability library").
Eg: PORTLIB=${TET_ROOT}/port/libport.a
TETINCDIR The directory containing the TET headers.
Eg: TETINCDIR=${TETBASE}/inc/posix_c
TETLIB The directory containing the TET library.
Eg: TETLIB=${TETBASE}/lib/posix_c
TCM The Test Control Manager. This is part ofthe TET. It is an object file that is linkedwith each test.
Eg: TCM=${TETLIB}/tcm.o
TCMCHILD The Test Control Manager. This is part ofthe TET. It is an object file that is linkedwith each program that is executed within atest by tet_exec().
Eg: TCMCHILD=${TETLIB}/tcmchild.o
APILIB The TET API library.
- 8 -
User Guide for the X Test Suite
Eg: APILIB=${TETLIB}/libapi.a
3.3.4 Configuration parameters for the X Test Suite Only the first two of these parameters require changingunless directories have been moved from their defaultlocations. XTESTHOST The name of the host on which test suiteclients are to be executed. This may be setto the value returned by a command which canbe executed using the PATH you have set onyour host, or may be set to a specific name.This is used to produce a resource file named.Xdefaults-$(XTESTHOST) in the test executiondirectory. The resource file is created whenbuilding the test for XGetDefault. Thisparameter is only used in the Makefile of thetest for XGetDefault.
Eg. XTESTHOST=‘hostname‘
Eg. XTESTHOST=‘uname -n‘
Eg. XTESTHOST=triton
XTESTFONTDIR The directory in which to install the testfonts.
Eg: XTESTFONTDIR=/usr/lib/X11/fonts/xtest
XTESTROOT The directory that is the root of the X TestSuite.
Eg: XTESTROOT=${TET_ROOT}/xtest
XTESTLIBDIR The directory containing libraries for theX Test Suite.
Eg: XTESTLIBDIR=${XTESTROOT}/lib
XTESTLIB The xtest library. This library containssubroutines that are common to many tests inthe X Test Suite.
Eg: XTESTLIB=${XTESTLIBDIR}/libxtest.a
- 9 -
User Guide for the X Test Suite
XSTLIB The X Protocol test
library. This library contains subroutines that are common
to many tests in the X Protocol section of the X Test
Suite.
Eg: XSTLIB=${XTESTLIBDIR}/libXst.a
XTESTFONTLIB The fonts library. This library containsfont descriptions that are common to manytests in the X Test Suite.
Eg: XTESTFONTLIB=${XTESTLIBDIR}/libfont.a
XTESTINCDIR The xtest header file directory. Thisdirectory contains headers that are local tothe X Test Suite.
Eg: XTESTINCDIR=${XTESTROOT}/include
XTESTBIN The xtest binary file directory. Thisdirectory contains utility programs that areused by X Test Suite.
Eg: XTESTBIN=${XTESTROOT}/bin
3.3.5 System Parameters Location of system libraries and include files. SYSLIBS Options to cause the C compiler to search anysystem libraries that are required for theX Test Suite that are not searched bydefault. This will probably include Xlib.
Eg: SYSLIBS=-lX11
If you wish to build the X Test Suite to makeuse of the XTEST extension, you will need toinclude the XTEST library and the X extensionlibrary (in that order).
Eg: SYSLIBS=-lXtst -lXext -lX11
Imake variables: ExtraLibraries XP_SYSLIBS Any system libraries that are needed, to linkthe X Protocol tests. This will include Xlib,
- 10 -
User Guide for the X Test Suite
since libXst.a (which is part of the test suite)
will include at least one call to XOpenDisplay.
Eg: XP_SYSLIBS=-lX11
Imake variables: ExtraLibraries SYSINC Any commands that should be given to the Ccompiler to cause all relevant system includefiles to be included. This will probablyinclude /usr/include/X11.
Eg: SYSINC=-I/usr/include/X11
3.3.6 C Compiler Directives Directives to the C compiler. Usually only the first fourparameters will need changing. The remainder are internallyused parameters, which are an amalgam of previously setparameters. COPTS Options to the C compiler.
Eg: COPTS=-O
Imake variables: DefaultCDebugFlags andDefaultCCOptions DEFINES Options required by the C compiler to set upany required defines. For example in strictANSI Standard-C systems you will need todefine _POSIX_SOURCE. Additionally on anX/Open conformant system it may be necessaryto define _XOPEN_SOURCE.
Eg: DEFINES=-D_POSIX_SOURCE
If there is no symbol NSIG defined in thesystem header file signal.h, then this has tobe supplied for use by the TET API. Itshould be the number of signal types on thesystem.
Eg: DEFINES=-D_POSIX_SOURCE -DNSIG=32
If you wish to build the X Test Suite to makeuse of the XTEST extension, you will need todefine XTESTEXTENSION. XTESTEXTENSION is only used when building the
- 11 -
User Guide for the X Test Suite
X Test Suite library.
Eg: DEFINES=-D_POSIX_SOURCE -DNSIG=32 -DXTESTEXTENSION
Imake variables: StandardDefines XP_DEFINES C compiler defines specific to the X Protocoltests. This can be set as DEFINES, but you can buildsupport for additional connection methodsbeyond TCP/IP, using the following defines,if XP_OPEN_DIS is XlibNoXtst.c (R4/R5XOpenDisplay emulation):
-DDNETCONN - Connections can also use DECnet3.
-DUNIXCONN - Connections can also use UNIX4 domain sockets.
Refer to your documentation for building and
installing Xlib on your platform. If XP_OPEN_DIS is one of XlibXtst.c orXlibOpaque.c then none of the defines listedabove will be required.
Eg: XP_DEFINES=-D_POSIX_SOURCE -DUNIXCONN
Imake variables: StandardDefines LINKOBJOPTS Options to give to the LD program to linkobject files together into one object filethat can be further linked.
Eg: LINKOBJOPTS=-r
INCLUDES Options to cause C compiler to search thecorrect directories for headers. This shouldnot need changing as it is just an amalgam ofother parameters.
____________________
4.
DEC and DECnet are registered trademarks of Digital
Equipment Corporation.
4.
UNIX is a registered trademark of UNIX System Laborato-
ries, Inc. in the U.S. and other countries.
- 12 -
User Guide for the X Test Suite
INCLUDES=-I. ${PORTINC} -I${TETINCDIR} -I${XTESTINCDIR} ${SYSINC}
CFLAGS Flags for the C compiler. This should notneed changing as it is just an amalgam ofother parameters. Note that CFLOCAL is notdefined in the configuration file; it isavailable for use in makefiles, to defineparameters that only apply to a particularcase. (It intentionally uses parenthesesrather than braces)
CFLAGS=$(CFLOCAL) $(COPTS) $(INCLUDES) $(DEFINES)
XP_CFLAGS Flags for the C compiler. This parameter isused by the X Protocol tests in the X TestSuite. This should not need changing as itis just an amalgam of other parameters.
XP_CFLAGS=$(CFLOCAL) $(COPTS) $(INCLUDES) $(XP_DEFINES)
LDFLAGS Flags used by the loader. This is needed onsome systems to specify options used whenobject files are linked to produce anexecutable.
Eg. LDFLAGS=-ZP
LIBS List of libraries. This should not needchanging as it is just an amalgam of otherparameters.
LIBS=${XTESTLIB} ${XTESTFONTLIB} ${APILIB} ${PORTLIB}
XP_LIBS List of libraries. This parameter is used bythe X Protocol tests in the X Test Suite.This should not need changing as it is justan amalgam of other parameters.
- 13 -
User Guide for the X Test Suite
XP_LIBS=${XSTLIB} ${XTESTLIB} ${XTESTFONTLIB} ${APILIB} ${PORTLIB}
XP_OPEN_DIS A choice of which code to build in the XProtocol library to make an X serverconnection. This must be set to one of threepossible values: XlibXtst.c Use this option only if your Xlibincludes post R5 enhancements to_XConnectDisplay ensuring maximumportable protocol test coverage. Theseenhancements include arguments to_XConnectDisplay to return authorisationdetails on connection. If you use thisoption when your Xlib does not have theseenhancements to _XConnectDisplay, theresults of running the X Protocol testswill be undefined. XlibOpaque.c You have a normal R4 Xlib or early R5Xlib which you cannot patch to includethe enhancements to _XConnectDisplay, andyou cannot emulate these by buildingXlibNoXtst.c, so only client-nativetesting can be done portably, and nofailure testing of XOpenDisplay can bedone. This option uses XOpenDisplay tomake the connection, from which the filedescriptor is recovered for our own use.XCloseDisplay shuts down the connection. XlibNoXtst.c As for XlibOpaque.c but you can use theR4/R5 connection emulation supplied.(Note: R4/R5 independent) This willensure maximum protocol test coverage butmay not be portable to all platforms. Reasons for not being able to buildXlibNoXtst.c might include: i) different interfaces to connection setupand connection read/write; ii) different access control mechanisms. Refer to your Xlib documentation for furtherdetails.
Eg. XP_OPEN_DIS=XlibOpaque.c
3.3.7 Pixel validation section This section defines a number of parameters that are usedonly when generating known good image files. These are notintended to be modified and need not be used when runningthe test suite. They are only used in the development
- 14 -
User Guide for the X Test Suite
environment at UniSoft when generating known good
image files.
3.4 Clean configuration parameters
The TET provides a scheme to execute a clean
tool, which removes previously built tests and object
files.
All clean configuration parameters are contained in a
configuration file that forms part of the TET
. The file
$TET_ROOT/xtest/tetclean.cfg
contains all the parameters that are needed to clean theX Test Suite. To save configuration effort, we have arranged that thebuild and clean configuration files may contain identicalparameter settings. Both files are needed, since the TETrequires both a default build and clean configuration file. Copy the build configuration file into the cleanconfiguration file:
cd $TET_ROOT/xtest
cp tetbuild.cfg tetclean.cfg
3.5 System dependent source files This section describes source files and include filesprovided in the X Test Suite which contain data, which youmay need to edit to reflect the system under test. 3.5.1 Host address structures The file xthost.c in the directory $TET_ROOT/xtest/src/libcontains three items, which you may need to edit to reflectthe system under test. These are all related to themechanisms provided by the X server under test to add, getor remove hosts from the access control list. These areonly used in the tests for those Xlib functions which use ormodify the access control list. The host access control functions use the XHostAddressstructure. You should refer to the Xlib documentation foryour system, to determine the allowed formats for hostaddresses in an XHostAddress structure. You may also findit helpful to refer to the X Window System Protocoldocumentation supplied with the X server under test. Thesection describing the ChangeHosts protocol request givesexamples of host address formats supported by many Xservers. The symbols FamilyInternet, FamilyDECnet,FamilyChaos and FamilyUname are defined on many systems inthe include files X.h and Xstreams.h. The X server undertest is not guaranteed to support these families, and maysupport families not listed here. You should find out whichfamilies are supported for the X server under test, byexamining the header files supplied with your system, andconsulting the documentation supplied with the X server.
- 15 -
User Guide for the X Test Suite
Some default declarations are contained in the
file, but there is no guarantee that they will work
correctly on your system.
The three items are as follows:
1. You should ensure that there is a declaration for an
array xthosts[] of at least 5 XHostAddress structures
containing valid family,length,address triples.
2. You should ensure that there is a declaration for an
array xtbadhosts[] of at least 5 XHostAddress structures
containing invalid family,length,address triples. You should
ensure that there is a declaration for an array xtbadhosts[]
of at least 5 XHostAddress structures containing invalid
family,length,address triples. If you cannot use the
supplied examples, the simplest way to do this is to use an
invalid family, which is not supported by the X server under
test, in each structure of the array.
3. You should ensure that there is a declaration for a
function samehost() that compares two XHostAddress
structures and returns True if they are equivalent. (It is
unlikely that the sample function will need modification -
no systems requiring modification have yet been
identified).
- 16 -
User Guide for the X Test Suite
4. Building the TET
The X Test Suite runs under the Test Environment Toolkit (
TET ).
This section of the User Guide tells you how to build and
install the supplied version of the TET .
The following instructions assume the use of a Bourne
shell.
The PATH variable should have the directory
$TET_ROOT/xtest/bin prepended to it.
PATH=$TET_ROOT/xtest/bin:$PATH
export PATH
4.1 The portability library The current version of the TET used by the X Test Suite isdesigned to run on a POSIX.1 system. Since many systems running the X Window System currently runon BSD based systems a portability library that emulates therequired routines using BSD facilities has been provided. This library is not part of the TET itself. The portability library source is kept in $TET_ROOT/port. The portability library may be useful in porting the X TestSuite to other environments as described below. Pleaserefer to the "Release Notes for the X Test Suite" fordetails of the target execution environments, and a list ofsystems to which it has already been ported. 4.1.1 Porting to a POSIX.1 system If your system conforms to POSIX.1 and has an ANSIStandard-C compiler then this library should not be builtand so this section can be skipped. The exception is that
putenv() may not exist on a POSIX.15 system, however it is
in the SVID6 and the XPG7, so in practice most non-BSDmachines will have this function. 4.1.2 Porting to a BSD system If the system is a standard BSD one, then the portability
____________________
5.
IEEE Std 1003.1-1990, Portable Operating System Interface
for Computer Environments
6.
System V Interface Definition, Issue 1, AT&T, Spring
1985.
7.
X/Open Portability Guide Issue 3, Volume 2: XSI System
Interface and Headers
- 17 -
User Guide for the X Test Suite
library can be used as it is; build it as
follows.
cd $TET_ROOT/port
pmake
4.1.3 Porting to other systems The portability library may be useful as a base for portingthe TET to other non-POSIX systems, however the portabilitylibrary is designed to run on a BSD system, and will notnecessarily build without change on other systems. The following routines are emulated for use under a BSDsystem, and these may be needed on other systems:
getcwd() getopt() putenv() sigaction()
sigaddset() sigdelset() sigemptyset() sigfillset()
sigismember() sigpending() sigprocmask() sigsuspend()
strchr() strcspn() strftime() strpbrk() strrchr()
strspn() strtok() toupper() upcase()
vsprintf() waitpid()
Only the features that are used by the X Test Suite areemulated. They are not meant to mimic completely thestandard behaviour. There is also an include directory $TET_ROOT/port/INC thatcontains header files that are required that are not foundon a BSD system. These files contain only items that areneeded for the X Test Suite, they are not designed toreplace completely the standard ones. To adapt the portability library to other systems, thefollowing hints may be found useful: • Examine the directory $TET_ROOT/port/INC. If thesystem already provides a standard conforming headerfile of the same name as one in the INC directory, thenremove the version from the INC directory. • The header files contain the bare minimum required tocompile the X Test Suite, and use BSD features. It maybe necessary to alter them to suit the local system.This applies particularly to signal.h. • It may be necessary to add other header files. • In the library Makefile remove any function that isalready provided by the system in a standard conformingform. • Examine the code of the remaining functions to makesure that they will work on the target system. 4.2 Building libraries and utilities There is a top level Makefile which can be used toautomatically perform a number of the following steps. Youshould still check through the User Guide and perform thesteps which need to be done manually. In particular you needto build and install the test fonts as described in thesection entitled "Compiling and installing the test fonts".
- 18 -
User Guide for the X Test Suite
The top level Makefile enables the following
steps to be performed:
Building the TET :
1. The Test Case Controller (TCC)
2. The API library
Building the X test suite libraries and utilities:
1. Building the X test suite library
2. Building the X Protocol library
3. Building the X test fonts library
4. Building the mc utility
5. Building the blowup utility
To use the top level Makefile, move to the top level
directory:
cd $TET_ROOT
Make the utilities and libraries with the command:
make
4.3 The Test Case Controller (TCC) Move to the directory containing the TCC source.
cd $TET_ROOT/tet/src/posix_c/tools
Make the TCC with the command:
pmake install
Note: the supplied version of the TCC assumes that the cputility on your system supports recursive copy using theoption -r. There are two occurrences of cp in the fileexec.c which use this option. In the X Test Suite, recursive copying is not required. If your system does not support this option, you can removethe use of this option in the source code before buildingthe TCC. If you do this you may not be able to use thesupplied TCC with other test suites. Alternatively, you can provide a shell script in thedirectory $TET_ROOT/xtest/bin which copies files using cpbut ignores any option -r. 4.4 The API library Move to the API library source directory.
cd $TET_ROOT/tet/src/posix_c/api
- 19 -
User Guide for the X Test Suite
Run the command
pmake install
which should produce the files libapi.a and the Test CaseManager files tcm.o and tcmchild.o.
5. Building the X test suite libraries and utilities 5.1 The X test suite library A library of common subroutines for the X Test Suite hassource in $TET_ROOT/xtest/src/lib. This is builtautomatically when building tests in the X Test Suite.Should it be required to build it separately for any reasonrun the command.
cd $TET_ROOT/xtest/src/lib
pmake install
The list of source files in this library is described in the"Programmers Guide". 5.2 The X Protocol library A library of common subroutines for the X Protocol tests inthe X Test Suite has source in $TET_ROOT/xtest/src/libproto.This is built automatically when building tests in theX Test Suite. Should it be required to build it separatelyfor any reason run the command.
cd $TET_ROOT/xtest/src/libproto
pmake install
The list of source files in this library is described in the"Programmers Guide". 5.3 The X test fonts library A library of common subroutines defining the characteristicsof the test fonts for the X Test Suite has source in$TET_ROOT/xtest/fonts. This is built automatically whenbuilding tests in the X Test Suite. Should it be requiredto build it separately for any reason run the command.
cd $TET_ROOT/xtest/fonts
pmake install
The list of source files in this library is described in the"Programmers Guide".
- 20 -
User Guide for the X Test Suite
Note that the directory
$TET_ROOT/xtest/fonts also contains the test fonts
themselves in bdf format, which must be compiled and
installed. Instructions for performing these steps are
included in the next section entitled "Compiling and
installing the test fonts".
5.4 Compiling and installing the test fonts
The X Test Suite contains a series of test fonts which are
used to test the correctness of the information returned by
the graphics functions in the X Window System. This is done
by comparing the information returned by those functions
with the expected font characteristics which are compiled
into the tests via the X test fonts library. The X test
fonts library is described in an earlier section of this
document.
There are seven test fonts whose descriptions are contained
in the files
xtfont0.bdf xtfont1.bdf xtfont2.bdf
xtfont3.bdf xtfont4.bdf xtfont5.bdf
xtfont6.bdf
These files are located in the directory$TET_ROOT/xtest/fonts. The manner in which fonts should be compiled and installedfor any particular X server is system dependent, and youshould refer to the instructions supplied with your releaseof the X Window System for details of how to do this. Some sample instructions are given here which may be usefulon many systems. These may not be appropriate for yoursystem, or they may need adaptation to work properly on yoursystem and so are provided only as a guide. 1. Move to the directory $TET_ROOT/xtest/fonts.
cd $TET_ROOT/xtest/fonts
2. Compile the seven bdf files into snf, pcf, or fbformat, as appropriate for your system.
pmake comp_snf
or
pmake comp_pcf
or
- 21 -
User Guide for the X Test Suite
pmake comp_dxpcf
or
pmake comp_fb
3. Copy the compiled fonts into the server font directory(the XTESTFONTDIR configuration parameter).
pmake install_snf
or
pmake install_pcf
or
pmake install_dxpcf
or
pmake install_fb
5.5 Building the mc utility The mc utility is used to generate test set source files andMakefiles from a template file, known as a dot-m file. Thefile naming scheme is described further in appendix B. Thefile formats are described further in the "ProgrammersGuide". The Makefiles and test set source files will be createdusing mc whenever test sets are built, if the dot-m file isfound to be newer than the source file or Makefile, or ifthese files do not exist. Build mc and install in the xtest bin directory as follows.
cd $TET_ROOT/xtest/src/bin/mc
pmake install
cd $TET_ROOT/xtest/src/bin/mc/tmpl
pmake install
5.6 Building the blowup utility The blowup utility is required for examining any incorrectimage files generated by the X server during a test run.
- 22 -
User Guide for the X Test Suite
Instructions for running the blowup program are
given in the section entitled "Examining image
files".
Build blowup and install in the xtest bin directory
as follows.
cd $TET_ROOT/xtest/src/pixval/blowup
pmake install
- 23 -
User Guide for the X Test Suite
6. Building the tests
6.1 Building tests using the
TET
The entire X Test Suite can be built by using the build mode
of the TCC . In this mode, the build
configuration parameters in the file
$TET_ROOT/xtest/tetbuild.cfg are used to build each
test set in the X Test Suite separately.
cd $TET_ROOT/xtest
tcc -b [ -s scenario_file ] [ -j journal_file ] [ -y string ] xtest all
-b This invokes the TCC in build mode. (If you have justfinished building the TCC from the csh, you willprobably have to rehash to get tcc in your path.) -s scenario_file This option builds the test sets in the named scenariofile. The default is a file named tet_scen in thedirectory $TET_ROOT/xtest. For more details refer tothe section entitled "Building modified scenarios usingthe TET". -j journal_file This option sends the output of the build to the namedjournal file. The default is a file named journal in anewly created sub-directory of $TET_ROOT/xtest/results.Sub-directories are created with sequential four digitnumbers, with the TCC flags (in this case "b")appended. The TCC will exit if the specified journalfile already exists, thus the journal file should berenamed or removed before attempting to execute theTCC. -y string This option only builds tests which include thespecified string in the scenario file line. This may beused to build specific sections or individual testsets. xtest This is the name of the test suite. It determines thedirectory under $TET_ROOT where the test suite is to befound. all This is the scenario name in the default scenario file$TET_ROOT/xtest/tet_scen. For more details refer tothe section entitled "Building modified scenarios usingthe TET" This will execute the TET build tool in the TETconfiguration variable TET_BUILD_TOOL (which is normallypmake), in each test set directory of the X Test Suite. The journal file should be examined to verify that the buildprocess succeeded. The report writer rpt cannot interpretthe contents of a journal file produced during the buildprocess. Note: If the TCC terminates due to receipt of a signal which
- 24 -
User Guide for the X Test Suite
cannot be caught, the TCC may
leave lock files in the test source directories. Subsequent
attempts to restart the TCC may give error
messages saying that a lock file was encountered. At this
point TCC may suspend the build. It may be
necessary to find and remove files or directories named
tet_lock before continuing.
6.1.1 Signal handling in the
TET
An interrupt signal (caused for example by typing the system
interrupt character on the controlling terminal) will cause
the TCC to abort the currently executing test
case. The journal file output records the fact that the test
case was interrupted.
Any other signal which can be caught by the
TCC causes it to terminate. By default, the
system suspend character will also cause the
TCC to terminate. If you wish to be able to
suspend the TCC , you can add the relevant
signals to the parameter SIG_LEAVE in the Makefile for the
TCC . Signals in this list will not be
caught, but will cause their default action. This is
explained further in the Test Environment Tookit Release
Notes.
6.2 Building, executing and cleaning tests using the
TET
Each test in the X Test Suite may be built, executed and
cleaned before the next test set in the scenario. This mode
of use has the advantage that the entire X Test Suite may be
executed, without necessarily building all the test sets in
advance. This mode of use has the disadvantage that you will
need to rebuild a test set before rerunning, which will take
considerably longer than when it is built in advance.
To do this, skip to the section entitled "Executing the
X Test Suite", and refer to the instructions in the
sub-section entitled "Building, executing and cleaning
tests using the TET "
6.3 Building modified scenarios using the
TET
6.3.1 Format of the scenario file
The TET uses a scenario file to determine
which test sets to build. The file
$TET_ROOT/xtest/tet_scen is the default scenario
file. The format is basically a scenario name starting in
column one, followed by list of test sets to be built (each
starting beyond column one). Only one scenario named
"all" is provided in the default scenario
file.
The names of the test sets are given relative to the
directory $TET_ROOT/xtest, and must commence with a leading
slash.
6.3.2 Modifying the scenario file
The file $TET_ROOT/xtest/tet_scen may be modified
by removing lines corresponding to test sets which are not
wanted. These will then simply not be built by the
TCC . Alternatively, unwanted lines may be
commented out by placing # in column one of a
line.
It is recommended that the supplied scenario file should be
saved if it is modified.
- 25 -
User Guide for the X Test Suite
6.3.3 Creating new scenario files
A new scenario file may be created in the directory
$TET_ROOT/xtest. The TCC will use
this scenario file instead of the file
$TET_ROOT/xtest/tet_scen if it is passed via the
-s option. For example
cd $TET_ROOT/xtest
tcc -b -s scenario_file [ -j journal_file ] [ -y string ] xtest all
6.4 Building tests without using the TET See section 11, entitled "Building, executing and reportingtests without using the TET". 6.5 Building tests in space-saving format It is possible to build the tests in the X Test Suite suchthat all the executable files in one section are links to asingle executable file. This normally allows a considerablereduction in the disc space requirements for the X TestSuite when fully built. Note that the names of the files built in space-savingformat are different to the names of the separate executablefiles built using the instructions in previous sections.There is nothing to prevent both sets of executables beingbuilt (although there is no value in this, and unnecessarydisc space will be consumed). 6.5.1 Building tests in space-saving format using the TET Before reading this section, read the section entitled"Building the tests using the TET". This gives anexplanation of the build mode of the TET, and the structureof scenario files. A scenario named linkbuild is provided in a scenario filenamed link_scen in the directory $TET_ROOT/xtest. Thisenables the TCC to build the space-saving executable filesand create all the required links for each test set in eachsection of the X Test Suite. The -y option allows aparticular space-saving executable for a single section tobe built. Execute the command:
cd $TET_ROOT/xtest
tcc -b -s link_scen [ -j journal_file ] [ -y string ] xtest linkbuild
This command will execute the TET build tool in the TETconfiguration variable TET_BUILD_TOOL (which is normallypmake), in the top level directory of each section of thetest suite. 6.5.2 Building tests in space-saving format without usingthe TET This section describes how to build the space-savingexecutable files for a particular section of the X Test
- 26 -
User Guide for the X Test Suite
Suite directly without using the
TET .
This can be simply done by calling pmake in the required
directory. For example, to build all the space-saving
executable files for section 5 of the X Test Suite, execute
the command:
cd $TET_ROOT/xtest/tset/CH05
pmake
- 27 -
User Guide for the X Test Suite
7. Executing the X Test Suite
Once you have built the X Test Suite as described in the
previous sections, work through the following sections to
execute the tests.
7.1 Setting up your X server
The first step is to ensure that the X server to be tested
is correctly set up.
7.1.1 Formal verification testing
A number of the tests within the X Test Suite can only give
reliable results if there is no window manager and no other
clients making connections to the X server. Thus, when
conducting formal verification tests, there should be no
window manager and no other clients connected to the X
server.
It is recommended that you close down and restart your X
server before a formal verification test run, in order to
ensure that results produced are repeatable and are not
affected by earlier tests, although this is not strictly
necessary.
You should switch off the screen saver if possible before
starting formal verification tests. This is because some X
servers implement the screen saver in a way which interferes
with windows created by test suite clients, which may cause
misleading results. If the screen saver cannot be switched
off, the time interval should be set so large as to prevent
interference with the tests.
You should also ensure that access control is disabled for
the server under test, so that the test suite can make
connections to the server. Also (if the X server allows
this) you should ensure that clients on the host system (as
specified in the build configuration parameter
XTESTHOST ) can modify the access control
list. Some X servers support the -ac option which disables
host-based access control mechanisms. If this option is
supported, you should use it.
7.1.2 Informal testing and debugging
Although no guarantee can be made that the tests within the
X Test Suite will give correct results if there are window
managers and other clients connected to the X server, it is
still possible to run many tests satisfactorily.
This section gives some guidelines which may be helpful in
running tests with a window manager present, and still
deriving correct results. The guidelines have been derived
from the experience gained during the development of the
tests.
Using these guidelines in connection with the instructions
in section 11, entitled "Building, executing and
reporting tests without using the TET ",
gives a rapid means to investigate the results of particular
tests in detail.
1. Set XT_DEBUG_OVERRIDE_REDIRECT=Yes in your execution
configuration file. This is described in more detail in the
next section.
- 28 -
User Guide for the X Test Suite
2. Do not raise any windows on top of those
created by running tests.
3. Avoid having any windows at position (0,0). Note that
some window managers such as tvtwm create their own
"root" window at position (0,0). This mainly
affects tests for section 8 of the X11R4 Xlib
specifications.
4. Be prepared to lose the input focus when tests are
running and don’t forcibly restore it. This mainly
affects tests for section 8 of the X11R4 Xlib
specifications.
7.2 Execute configuration parameters
The next step is to set up the execution configuration
file.
All execution configuration parameters are contained in a
configuration file that forms part of the TET
. This file should be edited to reflect the configuration of
the X server to be tested and the underlying operating
system on which Xlib is implemented. The file
$TET_ROOT/xtest/tetexec.cfg
contains all the parameters that are needed to execute theX Test Suite. The parameters are grouped in eight sectionswithin the configuration file. Numeric execution parameters may be specified in decimal,octal, or hexadecimal. Octal values must be a sequence ofoctal digits preceded by 0. Hexadecimal values must be asequence of hexadecimal digits preceded by 0x or 0X. 7.2.1 Configuration parameters defined by the TET TET_EXEC_IN_PLACE Setting this variable to False indicates that fileswill be executed in a temporary execution directory.Use of a temporary execution directory for each testenables parallel execution of the test suite againstmultiple servers. Setting this variable to True will give you improvedperformance if you are not attempting parallelexecution of the test suite against multiple servers.
Eg: TET_EXEC_IN_PLACE=False
TET_SAVE_FILES This indicates which files generated during executionof tests are to be saved for later examination. Thisline should not be altered.
Eg. TET_SAVE_FILES=Err*.err,*.sav
- 29 -
User Guide for the X Test Suite
7.2.2 Configuration Parameters for the X Test
Suite
The following parameters are used in many places in the X
Test Suite. These should be set to match the X server to be
tested and the underlying operating system on which Xlib is
implemented.
XT_DISPLAY
This should be set to a display string that can be passed to
XOpenDisplay, to access the display under test. It must
include a screen; all testing is done for a particular
screen.
Eg: XT_DISPLAY=:0.0
XT_ALT_SCREEN If the display supports more than one screen, thisparameter should be set to the number of a screen thatis different from that incorporated in the XT_DISPLAYvariable. Set to the string UNSUPPORTED if only one screen isavailable. Note that this should be a screen number, not a displaystring that can be passed to XOpenDisplay.
Eg: XT_ALT_SCREEN=1
XT_FONTPATH This should be set to a comma separated list that is avalid font path for the X server. It should include atleast the components of the default font path for the Xserver, enabling the cursor font to be accessed. Oneof the components must be the directory in which thetest fonts were installed (see the section entitled"Compiling and installing the test fonts"). This parameter will be used to set the font path forspecific test purposes which access the test fonts. Thefont path is restored on completion of the specifictest purposes.
Eg: XT_FONTPATH=/usr/lib/X11/fonts/xtest/,/usr/lib/X11/fonts/misc/
XT_SPEEDFACTOR This is a speedfactor which should be set to reflectthe relative delay in response of the underlyingoperating system and X server combined. Co-operatingprocesses which must synchronize allow a time delay inproportion to this speedfactor, to account forscheduling delays in the underlying operating systemand X server. This should be set to a number greaterthan or equal to one. There should be no need tochange the default unless the round trip time to the Xserver can be very long ( >15 seconds); in this case
- 30 -
User Guide for the X Test Suite
set this parameter to a value larger than the
maximum round trip time divided by 3.
Eg: XT_SPEEDFACTOR=5
XT_RESET_DELAY Specifies a delay time in seconds. Set this to be atime which is greater than or equal to the maximum timerequired by your server to reset when the last clientis closed. The test suite pauses for this timewhenever a connection is about to be opened and theserver may be resetting. The server may be resettingwhen the test case is entered (in startup()) as aresult of closing the last connection in the previoustest case. The server also resets in a few places inthe test for XCloseDisplay().
Eg. XT_RESET_DELAY=1
XT_EXTENSIONS Specifies whether you wish to test the extendedassertions which require the XTEST extension. Set thisto Yes if the XTEST extension is available on yoursystem, and you have configured the test suite to usethe XTEST extension, and you want to execute thesetests, otherwise set to No.
Eg. XT_EXTENSIONS=No
7.2.3 Configuration parameters for specific tests The following parameters are used to control one or morespecific test purposes in the X Test Suite. These should beset to appropriate values for the X server to be tested. These parameters may cause temporary changes in the settingsof the X server under test (such as the font path).Settings are restored on completion of the specific testpurposes. XT_VISUAL_CLASSES A space separated list of the visual classes that aresupported for the screen given by XT_DISPLAY. Eachvisual class is followed by a list of depths at whichthe class is supported (enclosed by brackets andseparated by commas with no spaces). Visual classesand depths that are supported only by other screensshould not be included. Note that this parameter is only used to check thecorrectness of the values returned by XMatchVisualInfoand XGetVisualInfo. Other tests which loop over visualsobtain the values by calling these functions.
- 31 -
User Guide for the X Test Suite
Eg. XT_VISUAL_CLASSES=StaticGray(8) GrayScale(8) StaticColor(8)
PseudoColor(8) TrueColor(8) DirectColor(8)
(This must be typed as one line.)
XT_FONTCURSOR_GOOD This specifies the number of a glyph in the defaultcursor font known to exist. XT_FONTCURSOR_GOOD+2should also be a glyph in the default cursor font.Neither of these should be the same as the X server’sdefault cursor.
Eg: XT_FONTCURSOR_GOOD=2
XT_FONTCURSOR_BAD This specifies the number of a glyph in the defaultcursor font known not to exist. If no such valueexists, set to UNSUPPORTED.
Eg: XT_FONTCURSOR_BAD=9999
XT_FONTPATH_GOOD This should be set to a comma separated list that is avalid font path for the X server. It should bedifferent from XT_FONTPATH. It need not contain thetest fonts.
Eg: XT_FONTPATH_GOOD=/usr/lib/X11/fonts/100dpi/,/usr/lib/X11/fonts/75dpi/
XT_FONTPATH_BAD This should be set to a comma separated list that is aninvalid font path for the X server. If you cannotdetermine a suitable value, set to UNSUPPORTED. Thereis no default value - by default, tests which use thisparameter will be reported as UNSUPPORTED.
Eg: XT_FONTPATH_BAD=/jfkdsjfksl
XT_BAD_FONT_NAME This should be set to a non-existent font name.
XT_BAD_FONT_NAME=non-existent-font-name
XT_GOOD_COLORNAME This should be set to the name of a colour which exists
- 32 -
User Guide for the X Test Suite
in the colour database for the X
server.
Eg: XT_GOOD_COLORNAME=red
XT_BAD_COLORNAME This should be set to the name of a colour which doesnot exist in the colour database for the X server.
Eg: XT_BAD_COLORNAME=nosuchcolour
XT_DISPLAYMOTIONBUFFERSIZE This should be set to a non-zero value (the valuereturned by XDisplayMotionBufferSize) if the X serversupports a more complete history of pointer motion thanthat provided by event notification, or zero otherwise.The more complete history is made available via theXlib functions XDisplayMotionBufferSize andXGetMotionEvents.
Eg: XT_DISPLAYMOTIONBUFFERSIZE=256
7.2.4 Configuration parameters for Display functions The following parameters are used to control one or moretest purposes for Xlib Display functions which are insection 2 of the X11R4 Xlib specifications. These should beset to match the display specified in the XT_DISPLAYparameter. Some of these parameters are specific to the particularscreen of the display under test. This is also specified inthe XT_DISPLAY parameter. Settings to these parameters will not cause any change inthe settings of the X server under test. Suitable values for most of these parameters can be obtainedfrom the output of the X11 utility xdpyinfo. XT_SCREEN_COUNT This parameter should be set to the number of screensavailable on the display as returned by XScreenCount.
Eg: XT_SCREEN_COUNT=2
XT_PIXMAP_DEPTHS A space separated list of depths supported by thespecified screen of the display that can be used forpixmaps.
Eg: XT_PIXMAP_DEPTHS=1 8
- 33 -
User Guide for the X Test Suite
XT_BLACK_PIXEL
This parameter should be set to the black pixel value of the
specified screen of the display.
Eg: XT_BLACK_PIXEL=1
XT_WHITE_PIXEL This parameter should be set to the white pixel valueof the specified screen of the display.
Eg: XT_WHITE_PIXEL=0
XT_HEIGHT_MM This parameter should be set to the height inmillimeters of the specified screen of the display.
Eg: XT_HEIGHT_MM=254
XT_WIDTH_MM This parameter should be set to the width inmillimeters of the specified screen of the display.
Eg: XT_WIDTH_MM=325
XT_PROTOCOL_VERSION This should be set to the major version number (11) ofthe X protocol as returned by XProtocolVersion.
Eg. XT_PROTOCOL_VERSION=11
XT_PROTOCOL_REVISION This should be set to the minor protocol revisionnumber as returned by XProtocolRevision.
Eg. XT_PROTOCOL_REVISION=0
XT_SERVER_VENDOR This should be set to the X server vendor string asreturned by XServerVendor.
Eg: XT_SERVER_VENDOR=MIT X Consortium
- 34 -
User Guide for the X Test Suite
XT_VENDOR_RELEASE
This should be set to the X server vendor’s release
number as returned by XVendorRelease.
Eg. XT_VENDOR_RELEASE=5000
XT_DOES_SAVE_UNDERS Set this to Yes if the specified screen of the displaysupports save unders (indicated by XDoesSaveUndersreturning True) otherwise set to No.
Eg. XT_DOES_SAVE_UNDERS=Yes
XT_DOES_BACKING_STORE Set this to the following value: 0 - the specified screen supports backing storeNotUseful 1 - the specified screen supports backing storeWhenMapped 2 - the specified screen supports backing store Always The way the specified screen supports backing store isindicated by the return value of XDoesBackingStore.
Eg. XT_DOES_BACKING_STORE=2
7.2.5 Configuration parameters for connection tests The following parameters are used to control one or moretest purposes for XOpenDisplay, XCloseDisplay andXConnectionNumber. These should be set to match the displayspecified in the XT_DISPLAY parameter and thecharacteristics of the underlying operating system. Settings to these parameters will not cause any change inthe settings of the X server under test. These parameters are not used when making connections to theX server in other tests. XT_POSIX_SYSTEM This may be set to Yes to indicate that the underlyingoperating system is a POSIX system. If this parameteris set to Yes, some extended assertions which describeimplementation dependent functionality will be testedassuming POSIX concepts.
Eg. XT_POSIX_SYSTEM=Yes
XT_DECNET Set this to Yes if clients can connect to the X serverunder test using DECnet. This will be used (on a POSIXsystem) in the tests for XOpenDisplay.
- 35 -
User Guide for the X Test Suite
Eg. XT_DECNET=No
XT_TCP Set this to Yes if clients can connect to the X serverunder test using TCP streams. This will be used (on aPOSIX system) in the tests for XOpenDisplay.
Eg. XT_TCP=Yes
XT_DISPLAYHOST Set this to the hostname of the machine on which thedisplay is physically attached. This will be usedinstead of XT_DISPLAY (on a POSIX system) in the testsfor XOpenDisplay which specifically test the hostnamecomponent of the display name. Note that this may not be the same as the machine onwhich the test suite clients execute (XTESTHOST).
Eg. XT_DISPLAYHOST=xdisplay.lcs.mit.edu
XT_LOCAL Set this to Yes if clients can connect to a local Xserver without passing a hostname to XOpenDisplay.This will be used (on a POSIX system) in the tests forXOpenDisplay. This is usually the case when the Xserver under test is running on the same platform asthe X Test Suite. When a hostname is omitted, the Xlibimplementation of XOpenDisplay can use the fastestavailable transport mechanism to make localconnections.
Eg. XT_LOCAL=No
7.2.6 Configuration Parameters which do not affect testresults There are a number of execution configuration parameterswhich can be used to reduce the size of the journal file, ordump out more information from the test suite. They willnot alter the behaviour of the tests or the test results. XT_SAVE_SERVER_IMAGE When set to Yes, the image produced by the server thatis compared with the known good image is dumped to afile with suffix ".sav" .
Eg: XT_SAVE_SERVER_IMAGE=Yes
- 36 -
User Guide for the X Test Suite
XT_OPTION_NO_CHECK
This may be set to Yes to suppress the journal file records
containing CHECK keywords. Refer to appendix
D for information on the contents of these
messages.
Eg: XT_OPTION_NO_CHECK=Yes
XT_OPTION_NO_TRACE This may be set to Yes to suppress the journal filerecords containing TRACE keywords. Refer to appendix Dfor information on the contents of these messages.
Eg: XT_OPTION_NO_TRACE=Yes
7.2.7 Configuration Parameters for debugging tests There are a number of execution configuration parameterswhich should not be set when performing verification testruns. These are intended for debugging purposes. Theseparameters may affect the behaviour of some test purposes ifthey are set to assist debugging. XT_DEBUG This may be set to a debugging level. A higher levelproduces more debugging output. Output is only producedby the test suite at levels 1, 2 and 3. Setting thisvariable to 0 produces no debug output, and 3 giveseverything possible (setting this variable to 3 cangive an enormous volume of output so you should not dothis when running large numbers of test sets).
Eg: XT_DEBUG=0
XT_DEBUG_OVERRIDE_REDIRECT When set to Yes, windows are created withoverride_redirect set. This enables tests to be runmore easily with a window manager running on the samescreen. This should not be set to Yes for verificationtests.
Eg: XT_DEBUG_OVERRIDE_REDIRECT=No
XT_DEBUG_PAUSE_AFTER When set to Yes, the test pauses after each call to theXlib function being tested, until Carriage Return isentered. This is useful to enable the results ofgraphics operations to be observed. This should not beset to Yes for verification tests.
- 37 -
User Guide for the X Test Suite
Eg: XT_DEBUG_PAUSE_AFTER=No
XT_DEBUG_PIXMAP_ONLY When set to Yes, tests which would normally loop overboth windows and pixmaps are restricted to loop overjust pixmaps. This is useful for speeding up theexecution of the test set. This should not be set toYes for verification tests. If XT_DEBUG_WINDOW_ONLY is also set to Yes, some testswill report UNRESOLVED due to the fact that nothing hasbeen tested.
Eg: XT_DEBUG_PIXMAP_ONLY=No
XT_DEBUG_WINDOW_ONLY When set to Yes, tests which would normally loop overboth windows and pixmaps are restricted to loop overjust windows. This is useful for speeding up theexecution of the test set. This should not be set toYes for verification tests. If XT_DEBUG_PIXMAP_ONLY is also set to Yes, some testswill report UNRESOLVED due to the fact that nothing hasbeen tested.
Eg: XT_DEBUG_WINDOW_ONLY=No
XT_DEBUG_DEFAULT_DEPTHS When set to Yes, tests which would normally loop overmultiple depths are restricted to test just the firstvisual returned by XGetVisualInfo and/or the firstpixmap depth returned by XListDepths (depending onwhether XT_DEBUG_PIXMAP_ONLY or XT_DEBUG_WINDOW_ONLY isalso set). This is useful for speeding up theexecution of the test set. This should not be set toYes for verification tests. Note that the first visual returned by XGetVisualInfomay not be the default visual for the screen.
Eg: XT_DEBUG_DEFAULT_DEPTHS=No
XT_DEBUG_VISUAL_IDS When set to a non-empty string, tests which wouldnormally loop over multiple depths are restricted totest just the visuals ID’s listed. Note that visualID’s for visuals on more than one screen may beentered, but those used will depend on whether the testbeing executed uses visuals on the default screen oralternate screen. The visuals ID’s should be enteredin decimal, octal or hexadecimal and separated with
- 38 -
User Guide for the X Test Suite
commas and with no intervening spaces. This
should not be set to a non-empty string for verification
tests.
Eg. XT_DEBUG_VISUAL_IDS=0x22,0x24,0x27
XT_DEBUG_NO_PIXCHECK When set to Yes, tests which would normally performpixmap verification omit this (all other processing isperformed in those tests as normal). Pixmapverification is a scheme which compares the imageproduced by the X server with a known good image filewhich is part of the X Test Suite (this is describedfurther in the section entitled "Examining ImageFiles"). This should not be set to Yes forverification tests.
Eg: XT_DEBUG_NO_PIXCHECK=No
XT_DEBUG_BYTE_SEX When set to NATIVE, REVERSE, MSB or LSB, the X Protocoltests will only be executed with the specified bytesex. When set to BOTH, the X Protocol tests makeconnections to the X server using both the native andreversed byte sex. Note: The parameter should always be set to NATIVE whenthe build configuration parameter XP_OPEN_DIS was setto XlibOpaque.c
Eg: XT_DEBUG_BYTE_SEX=NATIVE
XT_DEBUG_VISUAL_CHECK When set to a non-zero value, the X Protocol tests willpause for the specified time interval (in seconds), toenable a visual check to be performed on the displayedscreen contents.
Eg: XT_DEBUG_VISUAL_CHECK=5
7.2.8 Configuration Parameters used only during testdevelopment This section defines a number of parameters that are usedonly when generating known good image files. These are notintended to be modified and need not be used when runningthe test suite. They are only used in the developmentenvironment at UniSoft when generating known good imagefiles. XT_FONTDIR The directory in which the xtest fonts are located
- 39 -
User Guide for the X Test Suite
(before being installed). This must be set such
that appending a string gives a valid file name. This is
normally set to $TET_ROOT/xtest/fonts/.
Eg: XT_FONTDIR=/usr/mit/testsuite/xtest/fonts/
7.3 Executing tests using the TET The X Test Suite is executed by invoking the execute mode ofthe Test Case Controller.
cd $TET_ROOT/xtest
tcc -e [ -s scenario_file ] [ -j journal_file ] [ -x config_file ]
[ -y string ] xtest all
-e This invokes the TCC in execute mode. -s scenario_file This option executes the test sets in the namedscenario file. The default is a file named tet_scen inthe directory $TET_ROOT/xtest. For more details referto the section entitled "Executing modified scenariosusing the TET". -j journal_file This option sends the test results to the named journalfile. The default is a file named journal in a newlycreated sub-directory of $TET_ROOT/xtest/results.Sub-directories are created with sequential four digitnumbers, with the TCC flags (in this case "e")appended. The TCC will exit if the specified journalfile already exists, thus the journal file should berenamed or removed before attempting to execute theTCC. -x config_file This is an option to run the test suite using theinformation in a modified execution configuration filenamed config_file. The default is tetexec.cfg. -y string This option only executes tests which include thespecified string in the scenario file line. This may beused to execute specific sections or individual testsets. xtest This is the name of the test suite. It determines thedirectory under $TET_ROOT where the test suite is to befound. all This is the scenario name in the default scenario file$TET_ROOT/xtest/tet_scen. For more details refer tothe section entitled "Executing modified scenariosusing the TET". A journal file will be produced. More information on thecontents of the journal file is given in appendix C.
- 40 -
User Guide for the X Test Suite
Note: If the TCC terminates due to
receipt of a signal which cannot be caught, the
TCC may leave lock files in the test source
directories. Subsequent attempts to restart the
TCC may give error messages saying that a
lock file was encountered. At this point TCC
may suspend the build. It may be necessary to find and
remove files or directories named tet_lock before
continuing.
7.4 Building, executing and cleaning tests using the
TET
Each test in the X Test Suite may be built, executed and
cleaned before the next test set in the scenario. This mode
of use has the advantage that the entire X Test Suite may be
executed without necessarily building all the test sets in
advance, thus ensuring disc space is conserved throughout.
This mode of use has the disadvantage that you will need to
rebuild a test set before rerunning, which will take
considerably longer than if it is built in advance.
The X Test Suite is built, executed and cleaned by
simultaneously invoking the build, execute and clean modes
of the Test Case Controller.
cd $TET_ROOT/xtest
tcc -bec [ -s scenario_file ] [ -j journal_file ] [ -x config_file ]
[ -y string ] xtest all
-b This invokes the TCC in build mode. -e This invokes the TCC in execute mode. -c This invokes the TCC in clean mode. The other options are as described in the earlier sectionentitled "Executing tests using the TET". A journal file will be produced. This contains for each testset in order the results of the build, followed by the testresults, followed by the results of the clean. Moreinformation on the contents of the journal file is given inappendix C. The default journal file is named journal in a newly createdsub-directory of $TET_ROOT/xtest/results. Sub-directoriesare created with sequential four digit numbers, with the TCCflags (in this case "bec") appended. 7.5 Executing modified scenarios using the TET 7.5.1 Format of the scenario file The TET uses a scenario file to determine which test sets toexecute. The file $TET_ROOT/xtest/tet_scen is the defaultscenario file. The format is basically a scenario namestarting in column one, followed by list of test sets to beexecuted (each starting beyond column one). Only onescenario named "all" is provided in the default scenariofile. The names of the test sets are given relative to thedirectory $TET_ROOT/xtest, and must commence with a leadingslash.
- 41 -
User Guide for the X Test Suite
7.5.2 Modifying the scenario file
The file $TET_ROOT/xtest/tet_scen may be modified
by removing lines corresponding to test sets which are not
wanted. These will then simply not be executed by the
TCC . Alternatively, unwanted lines may be
commented out by placing # at the start of the
line.
If you wish to execute just a subset of the test purposes in
a test set, refer to the section below entitled
"Executing individual test purposes using the
TET ".
It is recommended that the supplied scenario file should be
saved if it is modified.
7.5.3 Creating new scenario files
A new scenario file may be created in the directory
$TET_ROOT/xtest. The TCC will use
this scenario file instead of the file
$TET_ROOT/xtest/tet_scen if it is passed via the
-s option. For example
cd $TET_ROOT/xtest
tcc -e -s scenario_file [ -j journal_file ] [ -x config_file ]
[ -y string ] xtest all
7.6 Executing individual test purposes using the TET Each assertion in the X Test Suite has separate test codewhich is known as a test purpose. We have arranged thateach test purpose is also a separately invocable component,and that the invocable component number is identical to thetest purpose number. The expression within the braces at the end of a line withina scenario file is an invocable component list (or IC_list).The default invocable component list all causes the TCC toexecute all invocable components in a test set. By altering the invocable component list for a test set,particular invocable components of interest can be executed. The invocable component list consists of one or moreelements separated by commas. Each element is either aninvocable component number, or a range of invocablecomponent numbers separated by a dash. This is useful for quickly executing a particular testpurpose of interest for example:
/tset/CH05/stclpmsk/Test{3}
This is also useful for executing all test purposes exceptone known to cause a system error. This may be useful if aparticular test purpose causes your X server to exit (atpresent the TET provides no high level control facilities toconditionally cancel later test sets). For example:
/tset/CH05/stclpmsk/Test{1-2,4-6}
- 42 -
User Guide for the X Test Suite
Note that the placement of windows used by the
test suite may differ when an earlier test purpose is not
executed. It is intended that test purposes produce the same
results regardless of window placement.
7.7 Executing tests without using the
TET
See section 11, entitled "Building, executing and
reporting tests without using the TET
".
7.8 Executing tests in space-saving format using the
TET
Before reading this section, read the section entitled
"Building tests in space-saving format". When you
have built all the sections of the test suite in
space-saving format, you can execute all the tests in the
test suite using the instructions in this section.
A scenario named linkexec is provided in a scenario
file named link_scen in the directory
$TET_ROOT/xtest. This enables the TCC to
execute the space-saving executable files which have been
built.
Execute the command:
cd $TET_ROOT/xtest
tcc -e -s link_scen [ -j journal_file ] [ -x config_file ]
[ -y string ] xtest linkexec
- 43 -
User Guide for the X Test Suite
8. Report writer
A basic report writer rpt is included with the X
Test Suite. It extracts and formats the main information
from a TET journal file produced by executing the
TCC in execute mode, or build-execute-clean
mode. It does not format the TET journal file
produced by the TCC in build only or clean
only mode. The main features of the TET
journal file produced by the TCC in execute
or build-execute-clean mode are described in appendix C.
Execute the report writer as follows:
rpt [ -t ] [ -d ] [ -p ] [ -s ] [ -f file ]
With only the -f argument, rpt lists the results of eachtest purpose for all test sets that appear in the journalfile file. The default is the file named journal in thehighest numbered subdirectory of the $TET_ROOT/xtest/resultsdirectory that has an ’e’ suffix. The reason for any test result code which is other than PASSis printed out. This is done by copying the testinformation messages of type REPORT. For further details,see appendix D. A warning message is printed if a test information messageof type REPORT is given in a test purpose which produced atest result code PASS. The results for each test set are followed by a summary ofthe number of test purposes in the test set which producedeach result code type. There is no overall summary list of results for all testsets in the journal file. -t Test information messages of type TRACE in the testpurposes specified are printed. For further details,see appendix D. -d Test information messages of type TRACE or DEBUG in thetest purposes specified are printed. For furtherdetails, see appendix D. -p Output is restricted to omit reporting on test purposesthat resolve to PASS, UNSUPPORTED, UNTESTED or NOTINUSE—thereby reporting only tests showing possible errors. -s The result summaries after the end of each test set areomitted.
- 44 -
User Guide for the X Test Suite
9. Examining image files
9.1 Generating pixmap error files
During the test run, discrepancies may be encountered
between the image displayed by the server and the known good
image. The known good image may have been obtained from a
known good image file supplied with the release, or it may
have been determined analytically.
Should a discrepancy be encountered, the test purpose will
give a result code of FAIL. The failure reason message will
name a pixmap error file in which is contained both the
known good image and the server image.
A debug option has been provided, which skips any
verification of the image produced by the server with known
good image files. This is done by setting the execution
configuration parameter XT_DEBUG_NO_PIXCHECK
to Yes.
9.2 Pixmap error file naming scheme
Each invocation of the TCC creates a
sub-directory in $TET_ROOT/xtest/results.
Sub-directories are created with sequential four digit
numbers, with the TCC flags ("e" or
"bec") appended. The default TET
journal is a file named journal created in this
directory.
Pixmap error files are stored in a directory tree created
within the newly created results sub-directory. So, for
example, when the line
/tset/CH06/drwln
is executed in a scenario file, pixmap error files might beproduced in a directory named$TET_ROOT/xtest/results/0001bec/tset/CH06/drwln. The creation of a new results directory tree for eachexecution of the TCC enables results to be obtained inparallel against multiple X servers. Pixmap error files are named Errnnnn.err, where nnnn is afour digit number. This number does not correspond to thenumber of the test purpose which caused the error. Note - when tests are executed without using the TCC theerror files are produced in the current directory. 9.3 Known good image file naming scheme All the required known good image files for the testprograms in the X Test Suite (as supplied) have been createdin advance. The known good image files for each testprogram are supplied in the X Test Suite in the test setdirectory in which the dot-m file is supplied. They arenamed annn.dat, where nnn is the number of the test purposefor which the known good image file was generated. More details of the contents of this release are in appendixA.
- 45 -
User Guide for the X Test Suite
9.4 Using blowup to view image files
The contents of the two images in a pixmap error file may be
compared by using the blowup utility.
Also, a known good image file may be viewed directly.
The file formats of the error file and the known good image
file are the same. The blowup utility detects which file
type is being viewed by means of file name. For this reason,
do not rename the pixmap error or known good image
files.
9.4.1 Blowup command
The blowup utility may be used to view one or more pixmap
error files or known good image files as
follows:
blowup [-z zoom_factor] [-f font] [-d display] [-colour] file(s)
-z zoom_factor This option sets the magnification factor in all theblowup windows. -f font This option ensures that font is used rather than thedefault font. The default font is 6x10. -d display This option uses the display named display for thedisplay windows. -colour On a colour display, this option will display differentpixel values in different colours corresponding to thatserver’s colour table. No attempt is made to preservecolours between different servers. 9.4.2 Blowup windows Two windows are created. The first is called Comparison, andthe second is called Blowup. The Blowup window shows amagnified version of a portion of the Comparison window,which is indicated in the Comparison window by a rectangle.A user interface menu is shown in the Blowup window. The title of the Comparison window will change to "ServerData", then to "Pixval Data" and then back to "Comparison"when the "B/G/Both" option on the menu is used. 9.4.3 Selection of a viewing region This may be done in one of three ways: 1. Click in the Comparison window. 2. Click in a square in the Blowup window. This becomesthe new centre square in that window. 3. Choose the "next error" option on the menu. The nextpixel at which there is a discrepancy will become thenew centre square in the Blowup window. 9.4.4 Information displayed The value stored in the centre pixel and its coordinates areshown as the top items in the menu. Under somecircumstances, the expected pixel value will be shown to theright of the actual value.
- 46 -
User Guide for the X Test Suite
9.4.5 Display of errors
When the "B/G/Both" option is set to Both, and the
title of the Comparison window is Comparison, errors are
displayed in two ways: one for each window.
In the Comparison window pixels set to non-zero in the
"good image" but set incorrectly in the
"server data" are shown as a cross (X).
In the blowup window these are shown as a white square with
a cross (X) through it.
In the Comparison window pixels set to zero in the
"good image" but set incorrectly in the
"server data" are shown as shaded squares.
In the blowup window these are shown as a black square with
a white cross through it.
The reason that we have proposed the two different methods
of displaying errors is as follows. One normally has a
higher magnification in the Blowup window and the use of a
cross (X) through incorrect pixels is good, and simple to
remember at this level of zoom. In the Comparison window
this style of display does not work well at the lower
magnification levels; all the crosses merge to a blur so it
is hard to see what type of error is being
displayed.
9.4.6 Commands (via menu in the Blowup
window)
All of the commands are invoked by clicking the left mouse
button when the corresponding menu item is highlighted
(inverted). The available commands are, from top to
bottom:
B/G/Both
Show Bad (Server Data), Good (Pixval Data) or Both
(Comparison). Clicking in this advances around the
cycle
Bad ----> Good -> Both
−-----<------<----/
The Comparison window’s name changes to reflect thecurrent state. If a known good image file is being displayed then onlythe Good option is available. A pixmap error file isrequired for this command to be useful. color/mono Use colour/monochrome in the Blowup window. next error Advance centre pixel point to be next pixel at whichthere is a discrepancy. sub-zoom + Zoom in (make bigger by zoomfactor) on the Blowupwindow sub-zoom - Zoom out (make smaller by zoomfactor) on the Blowupwindow quit Quit from the blowup utility. big-zoom + Zoom in (make bigger by zoomfactor) on the Comparisonwindow
- 47 -
User Guide for the X Test Suite
big-zoom -
Zoom out (make smaller by zoomfactor) on the Comparison
window
next
View next file in the list. The Blowup window will be
removed and a new one created for each file. The size, and
zoom factor, of the Comparison window will be preserved
across files.
- 48 -
User Guide for the X Test Suite
10. Cleaning the tests
10.1 Cleaning tests using the
TET
The entire X Test Suite can be cleaned by using the clean
mode of the TCC . In this mode, the clean
configuration parameters in the file
$TET_ROOT/xtest/tetclean.cfg are used to clean each
test set in the X Test Suite separately. Previously built
test set executables and object files are
removed.
cd $TET_ROOT/xtest
tcc -c [ -s scenario_file ] [ -j journal_file ] [ -y string ] xtest all
-c This invokes the TCC in clean mode. -s scenario_file This option cleans the test sets in the named scenariofile. The default is a file named tet_scen in thedirectory $TET_ROOT/xtest. For more details refer tothe section entitled "Cleaning modified scenarios usingthe TET". -j journal_file This option sends the output of the clean to the namedjournal file. The default is a file named journal in anewly created sub-directory of $TET_ROOT/xtest/results.Sub-directories are created with sequential four digitnumbers, with the TCC flags (in this case "c")appended. The TCC will exit if the specified journalfile already exists, thus the journal file should berenamed or removed before attempting to execute theTCC. -y string This option only cleans tests which include thespecified string in the scenario file line. This may beused to clean specific sections or individual testsets. xtest This is the name of the test suite. It determines thedirectory under $TET_ROOT where the test suite is to befound. all This is the scenario name in the default scenario file$TET_ROOT/xtest/tet_scen. For more details refer tothe section entitled "Cleaning modified scenarios usingthe TET". This will execute the TET clean tool in the TETconfiguration variable TET_CLEAN_TOOL (which is normallypclean), in each test set directory of the X Test Suite. The journal file should be examined to verify that the cleanprocess succeeded. The report writer rpt cannot interpretthe contents of a journal file produced during the cleanprocess. Note: If the TCC terminates due to receipt of a signal which
- 49 -
User Guide for the X Test Suite
cannot be caught, the TCC may
leave lock files in the test source directories. Subsequent
attempts to restart the TCC may give error
messages saying that a lock file was encountered. At this
point TCC may suspend the build. It may be
necessary to find and remove files or directories named
tet_lock before continuing.
10.2 Cleaning modified scenarios using the
TET
10.2.1 Format of the scenario file
Refer to the earlier section "Building modified
scenarios using the TET
".
10.2.2 Modifying the scenario file
Refer to the earlier section "Building modified
scenarios using the TET
".
10.2.3 Creating new scenario files
A new scenario file may be created in the directory
$TET_ROOT/xtest. The TCC will use
this scenario file instead of the file
$TET_ROOT/xtest/tet_scen if it is passed via the
-s option. For example
cd $TET_ROOT/xtest
tcc -c -s scenario_file [ -j journal_file ] [ -y string ] xtest all
10.3 Cleaning tests without using the TET See section 11, entitled "Building, executing and reportingtests without using the TET". 10.4 Cleaning tests built in space-saving format It is possible to clean the tests in the X Test Suite whichwere previously built in space-saving format. 10.4.1 Cleaning tests in space-saving format using the TET A scenario named linkbuild is provided in a scenario filenamed link_scen in the directory $TET_ROOT/xtest. Thisenables the TCC to clean the space-saving executable filesand remove all the required links for each test set in eachsection of the X Test Suite. The -y option allows aparticular space-saving executable and its links for asingle section to be removed. Execute the command:
cd $TET_ROOT/xtest
tcc -c -s link_scen [ -j journal_file ] [ -y string ] xtest linkbuild
This command will execute the TET clean tool in the TETconfiguration variable TET_CLEAN_TOOL (which is normallypclean), in the top level directory of each section of thetest suite.
- 50 -
User Guide for the X Test Suite
10.4.2 Cleaning tests in space-saving format without
using the TET
This section describes how to clean the space-saving
executable files for a particular section of the X Test
Suite directly without using the TET .
This can be simply done by calling pclean in the required
directory. For example, to clean all the space-saving
executable files for section 5 of the X Test Suite, execute
the command:
cd $TET_ROOT/xtest/tset/CH05
pclean
- 51 -
User Guide for the X Test Suite
11. Building, executing and reporting tests without
using the TET
11.1 Building tests
An individual test set can be rebuilt without the need to
use the build mode of the TCC . This is done
by executing pmake directly, rather than as a
TET build tool.
This is a useful facility for building a single test set
after a previous build has failed.
The build configuration parameters used by pmake
are obtained from a file named $TET_BUILDCONFIG, or, if
TET_BUILDCONFIG is not set in your environment, from the
file named $TET_ROOT/xtest/tetbuild.cfg.
The pmake command should be executed in the
directory containing the source code for the test set which
is to be rebuilt. For more details of the names of the
directories containing the source code for the test sets,
refer to appendix A.
For example
cd $TET_ROOT/xtest/tset/CH05/stclpmsk
pmake
No journal file is created when pmake is executed directly. The test set can also be rebuilt using the command
pmake Test
If there is a macro version of the Xlib function, this maybe rebuilt using the command
pmake MTest
11.2 Executing tests An individual test set can be executed without the need touse the execute mode of the TCC. This is done by executinga shell script pt. This is a useful facility for executing a single test setrepeatedly when investigating a particular test result. The execution configuration parameters used by pt areobtained from a file named $TET_CONFIG, or, if TET_CONFIG isnot set in your environment, from the file named$TET_ROOT/xtest/tetexec.cfg. The pt command is a shell script, which attempts to executethe binary file named Test in the current directory. If thefile Test is not found, the pt command attempts to executethe space-saving executable file built in that directory. The pt command should be executed in the directory
- 52 -
User Guide for the X Test Suite
containing the test set which has been built.
Unless you have manually installed the test set elsewhere,
this will be the directory containing the source code for
the test set. For more details of the names of the
directories containing the source code for the test sets,
refer to appendix A.
For example
cd $TET_ROOT/xtest/tset/CH05/stclpmsk
pt
A TET results file is created when pt is executed directly.This is a file named tet_xres located in the directory inwhich the test was executed. There are a number of options which may be passed to ptwhich alter the manner in which the test set is executed. Execute pt as follows:
pt [ -v XT_VARIABLE_NAME ] [ -d display ] [-i IC_list ] [ -p ] [ -w ]
[ -P ] [ -D ] [ -x debug_level ] [ -g ] [ -m ]
-v XT_VARIABLE_NAME=Value Modifies the value of the execution configurationparameter named XT_VARIABLE_NAME, assigning it a valueof Value. -d display Sets the display string to be used for the test. The default value is taken from the environmentvariable DISPLAY, or, if this is not set, from theexecution configuration parameter XT_DISPLAY. -i IC_list The invocable components executed will be thosespecified in IC_list. Each assertion in the X Test Suite has separate testcode, which is known as a test purpose. We havearranged that each test purpose is also a separatelyinvocable component, and that the invocable componentnumber is identical to the test purpose number. The invocable component list consists of one or moreelements separated by commas. Each element is either aninvocable component number or a range of invocablecomponent numbers separated by a dash. This is useful for quickly executing a particular testpurpose of interest for example:
pt -i 37
This is also useful for executing all test purposesexcept one known to cause a system error. For example:
- 53 -
User Guide for the X Test Suite
pt -i 1-36,38-57
Note that the placement of windows used by the testsuite may differ when an earlier test purpose is notexecuted. It is intended that test purposes produce thesame results regardless of window placement. -p This option is equivalent to setting the executionconfiguration parameter XT_DEBUG_PIXMAP_ONLY to Yes. -w This option is equivalent to setting the executionconfiguration parameter XT_DEBUG_WINDOW_ONLY to Yes. -P This option is equivalent to setting the executionconfiguration parameter XT_DEBUG_PAUSE_AFTER to Yes. -D This option is equivalent to setting the executionconfiguration parameter XT_DEBUG_DEFAULT_DEPTHS to Yes. -x debug_level This option is equivalent to setting the executionconfiguration parameter XT_DEBUG to debug_level. -g The binary file pvgen will be executed instead of thebinary file Test. This option should not be used, sincebinary files named pvgen are only used in thedevelopment environment at UniSoft when generatingknown good image files. -m The binary file MTest will be executed instead of thebinary file Test. Files named MTest contain tests forthe macro version of an Xlib function. Note that pt creates a temporary file CONFIG in the currentdirectory containing the configuration parameters, so writepermission is required to this file (or if no file is there,to the current directory). Note also that the binary file Test creates a temporary file.tmpresfd in the current directory containing theconfiguration parameters, so write permission is required tothis file. 11.3 Reporting tests The TET results file produced for an individual test set canbe formatted using the basic report writer rpt, which isdescribed in more detail in the section entitled "Reportwriter". The argument -f tet_xres formats the contents ofthe tet_xres file. For convenience, a separate report writer prp is provided,which is identical to rpt, except that the default file usedis tet_xres in the current directory. This is a useful facility for quickly formatting the resultsfrom the execution of a test set, and looking at the summaryof the result codes for each test purpose executed. The prp command should be executed in the directorycontaining the TET results file named tet_xres. Unless youhave manually installed and executed the test set elsewhere,this will be the directory containing the source code forthe test set. For more details of the names of the
- 54 -
User Guide for the X Test Suite
directories containing the source code for the
test sets, refer to appendix A.
For example
cd $TET_ROOT/xtest/tset/CH05/stclpmsk
prp
11.4 Cleaning tests An individual test set can be cleaned without the need touse the clean mode of the TCC. This is done by executingpclean directly, rather than as a TET clean tool. The clean configuration parameters used by pclean areobtained from a file named $TET_CLEANCONFIG, or, ifTET_CLEANCONFIG is not set in your environment, from thefile named $TET_ROOT/xtest/tetclean.cfg. The pclean command should be executed in the directorycontaining the test set which was built. For more details ofthe names of the directories containing the source code forthe test sets, refer to appendix A. For example
cd $TET_ROOT/xtest/tset/CH05/stclpmsk
pclean
No journal file is created when pclean is executed directly.
- 55 -
User Guide for the X Test Suite
12. Appendix A - Contents of X Version 11 Release
6.1
This section describes the contents of the directories in
the TET_ROOT directory which are supplied in this
release of the X Test Suite. The revised X Test Suite has
been developed from the T7 X Test Suite. This section
therefore also explains how the arrangement of the revised X
Test Suite compares with the T7 X Test Suite.
12.1 tet
This contains the source files and include files needed to
build the Test Environment Toolkit (TET). The contents of
the subdirectories are as follows:
12.1.1 tet/src
This contains the source files for the TET
.
12.1.2 tet/inc
This contains the include files for the TET
.
12.1.3 tet/lib
This contains the libraries and object files when the TET
has been built.
12.1.4 tet/bin
This should be empty since the TET utilities will be copied
into $TET_ROOT/xtest/bin, rather than this
directory, when using the modified Makefiles supplied with
the TET .
12.1.5 tet/doc
This contains the release notes and man pages for the
TET .
12.1.6 tet/demo
This contains a demonstration program for the
TET .
12.2 xtest
This contains the tests included in the revised X test suite
which are stored as a complete TET test
suite. This includes all necessary configuration files and
scenario files to enable you to use the TET
following the instructions in the
documentation.
12.3 xtest/bin
This contains commands you will need to install, configure,
build and execute the X Test Suite. After installation, this
directory contains shell script commands. After
configuration and building the X Test Suite, this directory
also contains executable programs built for your
system.
12.4 xtest/doc
This contains the documentation. It contains this user
guide, the programmers guide and the release notes. These
are supplied in troff(1) format requiring the
mm macro package, and also in PostScript format. It
also contains a template for error reports and a description
of how to submit them.
- 56 -
User Guide for the X Test Suite
It also contains a file paper.mm, which
is a copy of the file Xproto_verf/doc/paper.ms
originally supplied in the T7 X Test Suite, converted to use
the mm macro package. This file contains a paper
entitled "An Approach to Testing X Window System
Servers at a Protocol Level".
This is a technical paper, which defines in outline terms
the areas of the X Window System server which should be
tested at the X Protocol level rather than the Xlib
level.
The approach recommended in this paper, and adopted in the
design of the T7 X Test Suite, has been maintained in the
revised X Test Suite. The paper explains the choice of test
cases and division of tests between the X Protocol tests and
Xlib tests.
Before the revision of the X Test Suite, UniSoft recommended
that this paper should be left "as is". As a
result, some sections of this paper are out of date in that
they refer to development schedules for a previous software
development project, which have now been superseded with the
production of the revised X Test Suite.
12.5 xtest/fonts
This contains test fonts which should be installed using the
instructions in this user guide. It also contains a software
library describing the fonts which is used by the tests for
text drawing.
12.6 xtest/include
This contains include files for the software in the
xtest/src and xtest/tset directories.
12.7 xtest/lib
This contains libraries and other common software which are
used by the tests in the xtest/tset directory. The libraries
are built using the instructions in this user
guide.
12.8 xtest/results
This is an empty directory which is used by the
TET to store journal files produced when
executing the X Test Suite.
12.9 xtest/src
This contains the source for the libraries and utilities.
These are built using the instructions in this user
guide.
12.10 xtest/tset
This contains the source for the tests for sections 2 to 10
of the X11R4 Xlib specifications, in directories CH02 to
CH10. These are built using the instructions in this user
guide. In the rest of this document, these are refered to as
the "Xlib tests".
Each of the directories CH02 to CH10 contains further
subdirectories which are known as test set directories. Each
of these contain all the test code for a single Xlib
function. The name of the directory is derived from the name
of the Xlib function by a scheme which is described in
appendix B.
So, for example for XSetClipMask we have the
following:
- 57 -
User Guide for the X Test Suite
tset/CH05/stclpmsk
tset/CH05/stclpmsk/stclpmsk.m
The file stclpmsk.m is the source file, which is also knownas a dot-m file. The format of the dot-m file is describedfurther in the "Programmers Guide". The Xlib tests are designed to accomplish the following: 1. Test the ability of the Xlib function to behave asspecified in the X11R4 Xlib specifications, insituations where no X Protocol error should begenerated. This is tested by a series of separatetests known, as "test purposes", each of which isdesigned to test a single statement in the Xlibspecifications. The statement which is tested iscontained in an "assertion", which is also containedin the dot-m file, and precedes the test code for thattest purpose. 2. Test the ability of the Xlib function to produce theexpected X Protocol errors in specified situations.This is tested in further test purposes, each precededin the dot-m file by an assertion describing thesituation which should produce the error. 12.11 xtest/tset/XPROTO This contains the source for the touch tests for the XProtocol (version 11). These are built using theinstructions in this user guide. In the rest of thisdocument, these are refered to as the "X Protocol tests". These tests were in a separate test suite in the T7 X TestSuite, which was located in a directory Xproto_verf. Thisincluded separate documentation, drivers, parameter files,include files and libraries. In the revised X Test Suite,the directory XPROTO only contains the source of the tests -the other items are integrated within the X Test Suite asdescribed in this user guide. The directory XPROTO contains further subdirectories whichare known as test set directories. The structure of testset directories is exactly as for the Xlib tests, describedin the previous section. The X Protocol tests tests are designed to accomplish thefollowing: 1. Test the ability of the X Window System server toaccept all legal message types and respondappropriately. 2. Ensure that the server capabilities which Xlib testingdepends on work in the simplest cases. 3. Test that the X server adheres to the canonical bytestream of the X Protocol, independent of the host bytesex or compiler structure packing. For further details of the choice of test cases and divisionof tests between the X Protocol tests and Xlib tests, referto the document entitled "An Approach to Testing X WindowSystem Servers at a Protocol Level" contained in filextest/doc/paper.mm.
- 58 -
User Guide for the X Test Suite
13. Appendix B - File naming scheme
A file naming scheme has been devised which is for naming
the directories containing dot-m files and the dot-m files
themselves.
The file naming scheme converts from an X Window System name
to an abbreviated name. This is done as follows:
—Remove leading X.
—Replace:
Background -> Bg
Subwindow -> Sbwn
String -> Str
Window -> Wdw
—Remove all lowercase vowels aeiou.
—Truncate to 10 chars. We have already checked that
truncation to ten characters still gives uniqueness.
—If the string before truncation ended in
"16" then force truncated string to end in
"16".
—convert to lowercase.
—add ".m" suffix to get name of source file
containing C code, assertions and strategies.
- 59 -
User Guide for the X Test Suite
14. Appendix C - Format of the TET
journal file
This appendix describes the manner in which the X Test Suite
uses some of the TET journal file facilities.
The format of the TET journal file is not
fully described here - only the main features used by the X
Test Suite are described. In a future release of the X Test
Suite the format is expected to be described fully in the
TET documentation.
Journal files are produced by the TCC during
the build and execute stages.
The journal file produced during the execute stage contains
two basic sections:
1. Details of the configuration parameters and environment
in which the tests were executed. This may also be preceded
by build configuration parameters and/or followed by clean
configuration parameters, if the TCC was
invoked in build-execute-clean mode.
2. Details of the test results. This includes the test
result codes and test information messages output by the
test suite.
Each line in the second section of a journal file is made up
from three components separated by a vertical bar:
1. Message type. There is a unique numeric code for each
message type which is always the first field on a line.
2. Message parameters. These contain serial number and
similar information.
3. Message area. The format of this area is specific to the
Message Type.
An example of the second section is as follows:
10|53 /tset/CH02/vndrrls/vndrrls 13:41:12|TC Start, scenario ref 85-1, ICs {all}
15|53 1.9 1|TCM Start
520|53 0 7457 1 1|TRACE:NAME: XVendorRelease
400|53 1 1 13:41:14|IC Start
200|53 1 13:41:14|TP Start
520|53 1 7457 1 1|REPORT:XVendorRelease() returned 5000 instead of 1.
220|53 1 1 13:41:14|FAIL
410|53 1 1 13:41:14|IC End
80|53 0 13:41:15|TC End
This consists of a block of information for each test setexecuted which contains the following lines: 1. Message Type 10 - Test Case Start (TC Start) A single message indicating that the Test CaseController (the part of the TET which executes testsets) is about to execute a test set. This alsoindicates the start of the results for this test set(which in TET terminology is known as a test case).This line indicates the name of the test set obtainedfrom the scenario file. 2. Message Type 15 - Test Case Manager Start (TCM Start) A single message indicating that the Test Case Manager(the part of the TET which calls the test purposes)
- 60 -
User Guide for the X Test Suite
has started executing.
3. Each assertion in the X Test Suite has separate test
code, which is known as a test purpose. We have arranged
that each test purpose is also a separately invocable
component, and that the invocable component number is
identical to the test purpose number. For each test purpose
these lines follow:
1. Message Type 400 - Invocable Component Start (IC
Start)
The second field of the Message Parameters gives the IC
number.
2. Message Type 200 - Test Purpose Start (TP Start)
The second field of the Message Parameters gives the number
of the test purpose within the test set.
3. Message Type 520 - Test Case Information
The second field of the Message Parameters gives the number
of the test purpose within the test set.
The Message Area contains a text message output by the X
Test Suite - the possible types of message are described
further in appendix D "Interpreting test results"
in the section entitled "Test information
messages".
4. Message Type 220 - Test Purpose Result
The second field of the Message Parameters gives the number
of the test purpose within the test set.
The Message Area contains the test result code - the
possible test result codes are described further in appendix
D "Interpreting test results" in the section
entitled "Test result codes".
5. Message Type 410 - Invocable Component End (IC End)
The second field of the Message Parameters gives the IC
number.
4. Message Type 80 - Test Case End (TC End)
A single message indicating the end of the results for this
test set (which in TET terminology is known
as a test case).
- 61 -
User Guide for the X Test Suite
15. Appendix D - Interpreting test
results
This section includes information describing the
significance of the test result codes and the accompanying
test information messages that may appear in a
TET journal file.
15.1 Categorisation of assertions
The test result codes which are output for each test purpose
are dependent on the category of the assertion.
The model for the categorisation of assertions which is used
in the X Test Suite is described in POSIX.3.
There are four categories of assertions described by POSIX.3
which are designated A, B, C and D.
If the assertion tests a conditional feature, it is
categorised as type C or D, otherwise it is categorised as
type A or B.
If the assertion is classified as an "extended
assertion", it is categorised as type B or D. Otherwise
it is categorised as type A or C and is known as a
"base assertion".
Tests are always required for base assertions. Tests are not
required for extended assertions, but should be provided if
possible. There are a number of "extended
assertions" for which tests have been written in the X
Test Suite. Extended assertions are used to describe
features that may be difficult to test
conclusively.
Base Assertion Extended Assertion
Required Feature A B
Conditional Feature C D
15.2 Test result codes
The following test result codes may be found within the
TET journal file. These will be found in Test
Purpose Result lines with Message Type 220 (described in
appendix C).
The reason for any result codes NORESULT ,
UNRESOLVED and UNINITIATED
should be determined and resolved.
PASS The implementation passed the
test for the corresponding assertion.
- 62 -
User Guide for the X Test Suite
FAIL The implementation failed the
test for the corresponding assertion.
UNRESOLVED The test for the corresponding assertion
commenced, but was unable to establish the required
conditions to complete all required stages of the test. The
reasons for this should be investigated and if necessary the
test rerun.
In some tests, reliance is made on the successful behaviour
of another area of the X Window System. Where this reliance
is made, and that area of the X Window System does not
behave as expected, a result code UNRESOLVED
may occur. The test information messages should indicate the
area of the underlying problem. It may be necessary to look
at the test results for that area first and investigate and
resolve the underlying problem before re-running the
UNRESOLVED tests.
Tests which give a result code of UNRESOLVED and the message
"Path check error" normally contain programming
errors. The test reached the point at which a PASS result
would be assigned, but the number of check points passed in
executing the test code differs from the expected
number.
Tests which give a result code of UNRESOLVED and the message
"No CHECK marks encountered" may be due to
programming errors. The test reached the point at which a
PASS result would be assigned, but no check points had been
passed. This can also occur when you execute the tests using
debug options. For example, the message occurs when you
execute tests which normally loop over windows and pixmaps
and set XT_DEBUG_PIXMAP_ONLY=Yes or
XT_DEBUG_WINDOW_ONLY=Yes or
XT_DEBUG_DEFAULT_DEPTHS=Yes or
XT_DEBUG_VISUAL_IDS=Yes .
NOTINUSE Although there is an assertion within the
test set, there is no specific test provided for the
assertion. This might be because the assertion is tested
adequately as part of the test for another assertion, or
because the assertion has been automatically included into a
test set in which it is not applicable.
In either case, tests which report the result code NOTINUSE
do not need to be investigated further.
UNSUPPORTED This result code may only be used for
assertions in category C or D (conditional assertions).
The implementation does not support some optional feature of
the X Window System, which is needed to test the
corresponding
- 63 -
User Guide for the X Test Suite
assertion. In this case, the assertion will
normally make clear what optional feature is required, and
there will be an accompanying test information message
describing the feature which was found to be
unsupported.
UNTESTED This result code may only be used for
assertions in category B or D (extended assertions).
The implementation could not be conclusively tested to
determine the truth of the corresponding assertion.
Note that this does not mean that no testing was performed
in the X Test Suite. There are a number of "extended
assertions" for which we have provided tests where
possible (to test some likely problem areas, for
example).
UNINITIATED The test for the corresponding assertion
could not commence due to a problem in an earlier test
purpose or an initialisation routine.
Tests which produce this result code should be resolved as
for those reporting UNRESOLVED .
NORESULT Each test purpose should output a test
result code before completing. This special result code will
be inserted by the TET in the journal file,
if the test purpose did not output a test result code. This
indicates a major fault during the execution of the test set
which should be investigated.
WARNING The implementation passed the test for the
corresponding assertion. Whilst the behaviour of the
implementation was found to be acceptable, it behaves in a
manner which is not recommended in the X11 specification on
which the assertion is based.
FIP The contents of the journal file must be
examined by hand to determine whether the implementation
passed the test for the corresponding assertion. This is
used for testing functions which produce output whose
correctness cannot be easily determined automatically by the
test suite.
15.3 Test information messages
There are four types of test information messages which are
output by the X Test Suite. Each one results in a
TET journal file line with Message Type
"Test Case Information" (520), and with the
Message Area beginning with one of the following
keywords:
REPORT This keyword is used to
report the reason for any test result codes whch are other
than PASS . A warning message is printed by
the report writer rpt, if a test information
message of type REPORT is given in a test
purpose which produced a test result code
PASS .
- 64 -
User Guide for the X Test Suite
CHECK This keyword is used to
record the passing of a particular checkpoint in the test
suite code. These messages contain the checkpoint number
within the test purpose, and the line number within the
source code.
These messages are not output to the journal file if the
execution configuration parameter
XT_OPTION_NO_CHECK is set to Yes. This option
can reduce the size of the journal file
considerably.
TRACE This keyword is used for any messages
describing the state of the test being executed which are
not failure messages.
When running the X Protocol tests, messages with this
keyword are output to the journal file, to describe briefly
the interaction between the X server and the test
program.
These messages are not output to the journal file, if the
execution configuration parameter
XT_OPTION_NO_TRACE is set to Yes. This option
can reduce the size of the journal file
considerably.
DEBUG This keyword is used for debug messages
inserted during the development of the X Test Suite.
When running the X Protocol tests, messages with this
keyword are output to the journal file, when the debug level
is greater than zero, to describe in detail the interaction
between the X server and the test program. This includes the
contents of requests, replies and events.
This is output if the value of the execution configuration
parameter XT_DEBUG is greater than or equal
to the level of the debug message. XT_DEBUG
may be set from 0 to 3.
- 65 -
User Guide for the X Test Suite
16. Appendix E - Outline of Test Environment
Toolkit
The "Test Environment Toolkit" is a software tool
developed
by X/Open8, UNIX International (UI)9, and the Open Software
Foundation (OSF)10. The project which produced thissoftware and the associated interface specifications wasalso known as Project Phoenix. The TET consists of a user interface program known as theTest Case Controller (TCC). This enables test software to bebuilt and executed. The TCC uses configuration files tospecify the environment for both the build and executeoperations. The TCC also uses a scenario file to controlwhich tests to build or execute. The TCC produces a journal file which is intended to beformated by a test suite specific report writing program. The TET also includes an Application Programming Interface(API) Part of the API is the Test Case Manager (TCM). Thisincludes a main() function which calls user supplied testfunctions. The API also includes a library of functions tomanage the test functions and perform output operations tothe journal file in a structured fashion. Since the developers of the TET have indicated a commitmentto develop software test suites that execute within thisenvironment, the TET can be seen as an emerging de factostandard environment for test suite execution. During stage one of the X Test Suite development project weidentified that the TET provides features which are requiredby the revised test suite. For this reason we have developed the revised test suitewithin the TET environment, and supplied a copy of the TETwith the revised test suite. Release 1.9 of the TET was issued by the developers duringMarch 1992, and is included in this release of the X TestSuite. The software is complete in that the functionalityis stable and the implementation agrees with the TETspecification. Documentation including release notes andman pages for the TET utilities, are included in thisrelease of the X Test Suite. However, this release does notcontain a Programmers Guide or Users Guide. These are underdevelopment by UNIX International, but are not complete atthe time of this release, and so are not part of the current
____________________
8.
X/Open is a trademark of the X/Open Company, Ltd. in the
U.K. and other countries.
9.
UI is a trademark of UNIX International.
10.
Open Software Foundation, OSF and OSF/1 are trademarks of
the Open Software Foundation, Inc.
- 66 -
User Guide for the X Test Suite
version of the TET .
- 67 -
User Guide for the X Test Suite
17. Appendix F - Glossary
assertion
A statement of functionality for an element of the X
Window System, phrased such that it will be true for
a system conforming to the X Window System specifications.
An example would be a test description phrased
according to the requirements of POSIX.3.
assertion test
Synonymous with test purpose.
base assertion
An assertion for which a test suite must provide an
assertion test. Every assertion that is not an
extended assertion is a base assertion.
element
A particular X Window System interface such as an Xlib
function, header file or X11 Procotol.
extended assertion
An assertion for which a test suite is not required
to provide an assertion test. An assertion
test should still be provided if possible.
Reasons why a test suite is not required to provide a test
are given in appendix A of the Programmers Guide.
POSIX.1
Part one of the IEEE POSIX 1003 standards, the
document† entitled System Application Program
Interface (API) [C language]. Also known as P1003.1.
POSIX.3
Part three of the IEEE POSIX 1003 standards, the
document† entitled Test Methods for MeasuringConformance to POSIX. Also known as P1003.3. Project Phoenix Synonymous with TET. SVID System V Interface Definition. TCC The Test Case Controller. This is part of the TET. Thisis a user interface program which enables the testsoftware to be executed. See appendix E for moredetails. TCM The Test Case Manager. This is part of the TET. This isan object file containing a main() function which callsuser supplied test functions. See appendix E for moredetails. TET The "Test Environment Toolkit". This is a software toolwhich provides a framework within which tests may bedeveloped and executed. More information is given in
____________________
†
Obtainable from Publication Sales, IEEE Service Center,
P.O. Box 1331, 445 Hoes Lane, Piscataway, NJ 08854-1331,
(201) 981-0060
- 68 -
User Guide for the X Test Suite
appendix E.
test case
Synonymous with test set.
test description
The description of a particular test to be performed on an
element. This is presented in functional terms and
describes precisely what aspect of the X Window System is to
be tested for that element. An assertion is an
example of a test description, but the reverse is not
the case.
test program
Synonymous with test set.
test purpose
The software which tests the conformance of an
implementation of the X Window System to an
assertion.
test set
The software containing all the test purposes for an
element.
test strategy
A description of the design and method used to implement a
test purpose. This should say how a test
purpose is implemented rather than what feature is being
tested.
XPG
The X/Open Portability Guide.
Xlib tests
These are the tests for sections 2 to 10 of the X11R4 Xlib
specifications. They are stored in subdirectories of the
directories CH02 to CH10 (which are to be
found in the directory $TET_ROOT/xtest/tset).
X Protocol tests
These are the touch tests for the X Protocol (version 11).
They are stored in subdirectories of the directory
XPROTO (which is to be found in the directory
$TET_ROOT/xtest/tset).
CONTENTS
1.
Introduction
...................................... 1
2.
Preparation
....................................... 2
|
2.1 |
|
2.2 |
|
2.3 |
3.
Configuring the X Test Suite
...................... 4
|
3.1 |
|
3.2 |
|
3.3 |
|
3.4 |
|
3.5 |
4.
Building the TET
................................. 17
|
4.1 |
|
4.2 |
|
4.3 |
|
4.4 |
5.
Building the X test suite libraries and utilities
........................................ 20
|
5.1 |
|
5.2 |
|
5.3 |
|
5.4 |
|
5.5 |
|
5.6 |
6.
Building the tests
............................... 24
|
6.1 |
|
6.2 |
|
6.3 |
|
6.4 |
|
6.5 |
7.
Executing the X Test Suite
....................... 28
|
7.1 |
|
7.2 |
|
7.3 |
|
7.4 |
|
7.5 |
|
7.6 |
|
7.7 |
|
7.8 |
8.
Report writer
.................................... 44
9.
Examining image files
............................ 45
|
9.1 |
|
9.2 |
|
9.3 |
i
9.4
Using blowup to view image files
.......... 46
10.
Cleaning the tests
............................... 49
|
10.1 |
|
10.2 |
|
10.3 |
|
10.4 |
11.
Building, executing and reporting tests without using the
TET
.................................... 52
|
11.1 |
|
11.2 |
|
11.3 |
|
11.4 |
12.
Appendix A - Contents of X Version 11 Release 6.1
......................... 56
|
12.1 |
|
12.2 |
|
12.3 |
|
12.4 |
|
12.5 |
|
12.6 |
|
12.7 |
|
12.8 |
|
12.9 |
|
12.10 |
|
12.11 |
13.
Appendix B - File naming scheme
.................. 59
14.
Appendix C - Format of the TET journal
file
...... 60
15.
Appendix D - Interpreting test results
........... 62
|
15.1 |
|
15.2 |
|
15.3 |
16.
Appendix E - Outline of Test Environment Toolkit
.......................................... 66
17.
Appendix F - Glossary
............................ 68
ii