Xlib and X Protocol Test Suite
X Version 11 Release 6.1
Programmers Guide for the X Test Suite
July 1992
Copyright © 1991, 1992
UniSoft Group Limited
Permission to use, copy, modify, distribute, and sell this
software and its documentation for any purpose is hereby
granted without fee, provided that the above copyright
notice appear in all copies and that both that copyright
notice and this permission notice appear in supporting
documentation, and that the name of UniSoft not be used in
advertising or publicity pertaining to distribution of the
software without specific, written prior permission. UniSoft
makes no representations about the suitability of this
software for any purpose. It is provided "as is"
without express or implied
warranty.
Programmers Guide for the X Test Suite
Programmers Guide for the X Test Suite
1. Introduction
This document is a Programmers Guide to the X Test
Suite.
Instructions for installing and running the X Test Suite are
contained in the document "User Guide for the X Test
Suite". It is not necessary to read the Programmers
Guide in order to install and run the test
suite.
2. Purpose of this guide
The information in this section is designed to be used by a
programmer intending to review the source code in the
revised X Test Suite. It is also intended to be used by an
experienced programmer, familiar with the X Window System,
to modify or extend the X Test Suite to add additional
tests.
Before reading this document, it is necessary to read the
document "User Guide for the X Test Suite". This
is because the nomenclature used in this document is
explained in the "User Guide". Appendix F of that
document is a glossary, which explains the meaning of some
terms which may not be in common usage.
The directory structure used within the X Test Suite is
described further in "Appendix A - Contents of X
Version 11 Release 6.1" in the "User Guide".
You should be familiar with that appendix before reading
further.
3. Contents of this guide
The test set source files in the revised X Test Suite have
been developed in the format of a simple language, specially
produced as part of the test suite development project.
Files in the test suite which use this language have the
suffix ".m" and are known as dot-m files.
The syntax of this language is described in the next section
of this document entitled "Source file
syntax".
During the stage one review of the development project, it
was determined that there were some advantages in methods of
automatically generating source code for the tests from
templates. The dot-m file may be seen as a template for the
tests. At the same time, it was important that any utilities
associated with the code generation should be commonly
available or provided within the test suite.
For this reason, a utility mc has been provided to
convert dot-m files into C source code and produce Makefiles
automatically for the test set. The file formats which may
be produced are described in the section entitled
"Source file formats". Summaries of usage of the
utilies are given in appendices D-F.
As part of the test suite development, a number of
conventions were established to define how the syntax of
this language should be used in writing test sets. It is
useful to understand these conventions in order to
understand the structure of the existing tests. You
are
- 1 -
Programmers Guide for the X Test Suite
recommended to use the existing tests as a model,
and follow the same structure when modifying or extending
the X Test Suite. This is described in the section entitled
"Source file structure".
The test set structure has deliberately been kept as simple
as possible, and common functions have been developed in
libraries. The contents of these libraries is described in
the section entitled "Source file
libraries".
3.1 Typographical conventions used in this
document
The following conventions have been used in this
document:
1. Items appearing in angle brackets <> should be
substituted with a suitable value.
2. Items appearing in square brackets [] are
optional.
- 2 -
Programmers Guide for the X Test Suite
4. Source file syntax
It is a design requirement that the test code for each test
purpose has an associated description of what is being
tested (an assertion) and a description of the
procedure used to test it (a test strategy). All of
these items are contained in dot-m files, and programs are
used to extract the relevant parts. The utilies developed
for the purpose are outlined in appendices D-F, and the
format of the files they output is described in the section
entitled "Source file formats.
The format of a dot-m file consists of a number of sections
introduced by keywords. The sections of a dot-m file are as
follows:
1. A copyright notice.
2. A section introduced by the >>TITLE keyword. This
section defines the name of the function being tested and
its arguments.
The >>TITLE keyword, and the declaration and arguments
which follows it, are together known as the "title
section".
3. Optionally, there may be a section introduced by the
>>MAKE keyword. This section defines additional rules
for make beyond the default rules.
The >>MAKE keyword, and the text which follows it, are
together known as a "make section".
4. Optionally, there may be a section introduced by the
>>CFILES keyword. This section defines additional C
source files source files that should be compiled and linked
(together with the C source files produced by mc)
when building a test set.
5. Optionally, there may be a section introduced by the
>>EXTERN keyword. This section defines C source code
which will be in scope for all test purposes in the test
set.
The >>EXTERN keyword, and the source code which
follows it, are together known as an "extern
section".
6. For each test purpose in the test set, there will be a
section introduced by an >>ASSERTION keyword. These
sections each contain the text of an assertion for the
function being tested.
The >>ASSERTION keyword, and the text which follows
it, are together known as an "assertion
section".
7. An assertion section is normally followed by a section
introduced by a >>STRATEGY keyword, which is followed
by the strategy. This is a description of how the assertion
is tested.
The >>STRATEGY keyword, and the strategy which follows
it, are together known as a "strategy
section".
8. The strategy is always followed by a section introduced
by a >>CODE keyword, which is followed by a section of
C source code which will test whether the assertion is true
or false on the system being tested.
The >>CODE keyword, and the C source code which
follows it, are together known as a "code
section".
- 3 -
Programmers Guide for the X Test Suite
9. Optionally, the >>INCLUDE keyword
includes, from a file, one or more of the above sections, at
that point in the dot-m file. It terminates the previous
section.
The keywords introducing the sections are defined in detail
below.
There are also optional keywords which may appear anywhere
in a dot-m file:
1. The >>SET keyword sets options which apply in the
dot-m file.
2. The >># keyword introduces a comment in a dot-m
file.
The optional keywords are also defined in detail
below.
4.1 Title section - >>TITLE
4.1.1 Description
>>TITLE <function> <section>
This keyword must be used at the start of a test purposeimmediately after the copyright message. It may be followed by the declaration of the data typereturned and the function arguments. These lines may beomitted if XCALL is never used in any code section in thetest set. 1. The line after >>TITLE should specify the data typereturned. 2. The next line should specify the function call withany arguments. This is no longer mandatory, since theinformation is not used in the current version of mc.So the second line may be left completely blank. 3. The following lines should specify the arguments andmay set them to any expression (not necessarily aconstant expression). Usually arguments are set tothe return value of a function call, a global variableinserted by the mc utility, or a global variabledeclared in an >>EXTERN section. Static variables will be created to match thesearguments, and will be reset as specifiedautomatically before the start of each test purpose,by the function setargs() (described in the sectionentitled "Source file formats"). 4.1.2 Arguments function This is the name of the function in the X WindowSystem to be tested. In the X Protocol tests itshould be the name of an X Protocol request oran X event. In the Xlib tests it should be thename of an Xlib function or an X event. The XCALL macro will invoke the named functionor macro (which should be in Xlib) with thearguments specified on the lines following>>TITLE. section This is the section of the X Test Suite in whichthis particular test set is stored. It should
- 4 -
Programmers Guide for the X Test Suite
be the name of a directory in
$TET_ROOT/tset. The section name is used in
formatting the assertions in the test set, and for output to
the journal file for reporting purposes. The section name
does not affect the building and execution of the
tests.
4.2 Make section - >>MAKE
4.2.1 Description
>>MAKE
This keyword may be used anywhere in the dot-m file afterthe end of a section. It should be followed by lines which will be copied into theMakefile by the mc utility, when the Makefile is remadeusing the -m option. The lines specified will be joined together and placed inorder before the first rule in the Makefile. In this way,the lines copied may contain both initialisation of Makevariables, and additional rules. This keyword should be used sparingly, since the additionalMakefile lines must be consistent with the rest of theMakefile (see later section entitled "Makefile"). These lines are designed to allow auxiliary programs to bemade by make (in addition to the default target which isTest). For example, if test purposes in the test setexecute programs Test1 and Test2, which must also be built,the >>MAKE keyword can be followed by lines which specifyrules for building these executable programs. This can bedone as follows:
>>MAKE
AUXFILES=Test1 Test2
AUXCLEAN=Test1.o Test1 Test2.o Test2
all: Test
Test1 : Test1.o $(LIBS) $(TCMCHILD)
$(CC) $(LDFLAGS) -o $@ Test1.o $(TCMCHILD) $(LIBLOCAL) $(LIBS) $(SYSLIBS)
Test2 : Test2.o $(LIBS) $(TCMCHILD)
$(CC) $(LDFLAGS) -o $@ Test2.o $(TCMCHILD) $(LIBLOCAL) $(LIBS) $(SYSLIBS)
Notice that the default target Test is still made by the newdefault Make rule. Also, notice that Test1 and Test2 arenot dependencies for the all target. They are dependenciesfor Test. (See the later section entitled "Makefile".)
- 5 -
Programmers Guide for the X Test Suite
4.3 Additional source files - >>CFILES
4.3.1 Description
>>CFILES <filename> ...
This keyword may be used anywhere in the dot-m file afterthe end of a section. The list of files following the keyword are taken as thenames of C source files that should be compiled and linked(together with the C source files produced by mc) whenbuilding a test set. This allows code to be split amongseveral files. The only effect is to alter the Makefile that is produced bymmkf. 4.4 Extern section - >>EXTERN 4.4.1 Description
>>EXTERN
This keyword may be used anywhere in the dot-m file afterthe end of a section. It should be followed by lines of C source code which willbe copied into the C source file by the mc utility, when theC source code is remade. These lines will be copied unaltered into the C source filebefore the source code for the first test purpose. This section is useful for including three types of sourcecode: 1. static variables declarations which are used by anumber of test purposes in the test set. 2. static functions which are used by a number of testpurposes in the test set. 3. header file inclusions which are needed in addition tothe default header file inclusions in the C sourcecode. 4.5 Assertion section - >>ASSERTION 4.5.1 Description
>>ASSERTION <test-type> [ <category> [<reason>] ]
This keyword is used at the start of each test purpose. It should be followed by the text of the assertion which isa description of what is tested by this particular testpurpose. The text should not contain troff font change commands (orany other nroff/troff commands). This is because the
- 6 -
Programmers Guide for the X Test Suite
majority of nroff/troff commands will not be
understood by the ma utility. However, various
macros can be used to enable mapping of the format of
special text onto similar fonts to those used in the X
Window System documentation. These are described in appendix
B.
The keyword xname in the assertion text will be replaced by
mc with the name of the function under test
obtained from the >>TITLE keyword.
Unless the category argument specifies an extended
assertion, or the test-type is gc or
def, the assertion text must be followed by the
>>STRATEGY keyword, strategy section, >>CODE
keyword and code section. Refer to the description of the
category argument.
If the >>CODE keyword is missing, following the text
of an assertion which is not an extended assertion, the
mc utility will insert code to produce a result
code UNREPORTED for this test purpose when the test set is
executed.
4.5.2 Arguments
test-type
Good
This is a "good" test. The function under test is
expected to give a successful result.
By convention these assertions appear in the dot-m file
before all assertions with test-type Bad.
Bad
This is a "bad" test. The function under test is
expected to give an unsuccessful result under the conditions
that the test imposes.
By convention these assertions appear in the dot-m file
after all assertions with test-type Good.
The assertion text for many of these assertions is included
via the .ER keyword, described below.
gc
The assertion text states which gc components affect the
function under test. In this case the remaining arguments
are unused, and mc inserts into the C source file a
series of assertions, strategies and test code corresponding
to the gc components listed in the assertion text via the
macro
.M gc-comp ,
or
.M gc-comp .
- 7 -
Programmers Guide for the X Test Suite
The assertions, strategies and test code are
included, from files in the directory
$TET_ROOT/xtest/lib/gc.
The .M macro is used, since the gc components correspond to
structure members in a gc structure.
def
The assertion text is tested in the test for another
assertion. These assertions are often definitions of terms,
which cannot be tested in isolation, hence the abbreviation
"def". The remaining arguments are unused, and
mc inserts code into the C file to issue the result
code NOTINUSE, and issues a message stating that the
assertion is tested elsewhere.
category
This is the assertion category, modelled on the
corresponding codes in the document1 POSIX.3entitled "Test Methods for Measuring Conformanceto POSIX". It is either A, B, C or D. If the assertion tests a conditional feature, itis categorised as type C or D, otherwise it iscategorised as type A or B. If the assertion is classified as an "extendedassertion" it is categorised as type B or D.Otherwise it is categorised as type A or C andis known as a "base assertion".
Base Assertion Extended Assertion
Required Feature A B
Conditional Feature C D
Tests are always required for base assertions.
Tests are not required for extended assertions, but should
be provided if possible. Extended assertions are used to
describe features that may be difficult to test.
In some cases partial testing may be performed for extended
assertions. An example is that it may be possible to test
that some specific common faults are not present. In this
the result code would be FAIL if an error is detected, or
UNTESTED if no failure is detected, but the assertion is
still not fully tested.
For this reason, the strategy and code sections are optional
for extended assertions. If they are not supplied,
mc will automatically generate source code to put
out a result code UNTESTED, with a message which describes
the reason. If they are supplied, they will override the
automatically generated sections.
Since there is not yet an equivalent document to POSIX.3 for
the X Window System then these codes are subject to change.
For example, an assertion classified as an "extended
assertion" (type B) might become a "base
assertion" (type A) if a test method is later
identified.
The following table lists the allowed test result codes for
each category.
Category Allowed Result Codes
A PASS, FAIL, UNRESOLVED
B PASS, FAIL, UNTESTED, UNRESOLVED
C PASS, FAIL, UNSUPPORTED, UNRESOLVED
D PASS, FAIL, UNSUPPORTED, UNTESTED, UNRESOLVED
reason
In the case of extended assertions (category B or D) a
reason code must be supplied. These are the same as in
POSIX.3. A list of the reason codes, and the corresponding
text of the reason, are shown in appendix A.
4.6 Strategy section - >>STRATEGY
4.6.1 Description
>>STRATEGY
If the category of an assertion is A or C, this keyword mustbe used immediately after the assertion section. If the category of an assertion is B or D, this keyword maybe optionally be used immediately after the assertionsection. It should be followed by the strategy, which is adescription of how the assertion is to be tested. The text of the strategy is in free format sentences. Itmay contain the xname keyword, which is an abbreviation forthe name of the function under test. The use of the XCALL keyword in the strategy section as anabbreviation for "Call xname" is discontinued, although inthis release, some occurrences remain. 4.7 Code section - >>CODE 4.7.1 Description
>>CODE [<BadThing>]
This keyword must be used immediately after the strategysection. It should be followed by the C source code which will testthe assertion. The C source code will be converted by mc into a formatsuitable for the use by the TET API. The way in which thisis done is described in the section entitled "Source fileformats". A blank line must separate the declarations of automaticvariables in the test function from the first executablestatement. There must be no other blank lines within thesedeclarations. The utility mc also expands the XCALL macro to call thefunction under test, and to call library functions, beforeand after the function under test, to install and deinstallerror handlers and flush pending requests to the X server.The way in which this is done is described in the sectionentitled "Source file formats". 4.7.2 Arguments BadThing This is an optional argument to the >>CODEmacro. If it is omitted, the XCALL macro will causecode to be inserted to ensure that the functionunder test produces no X Protocol error, andissues a result code FAIL if an error isdetected. If it is set to the symbolic value of anX Window System error code, the XCALL macro willcause code to be inserted to ensure that thefunction under test produces that X Protocolerror, and issues a result code FAIL if thewrong error, or no error, is detected. 4.8 Included section - >>INCLUDE 4.8.1 Description
>>INCLUDE <filename>
This keyword includes the contents of filename, which mustbe a file containing one or more sections in the dot-m fileformat, optionally containing a copyright header. Thesections in the included file should produce a valid dot-mfile, if they were included directly at the point ofinclusion. The >>INCLUDE keyword terminates the precedingsection - it cannot be used in the middle of a section. The included sections are processed at the point ofinclusion when the C source code is generated by the mcutility. The >>INCLUDE keyword is usually used when including testpurposes which are common to more than one test set. The>>INCLUDE mechanism allows an entire test purpose (includingassertion, strategy and code sections) to be included. The >>INCLUDE keyword should not be used for merelyincluding common functions called by a number of testpurposes. If the functions are common to one dot-m file,they should be placed in an extern section. If the functionsare common to many test sets, they should be placed in oneof the X Test Suite libraries.
Programmers Guide for the X Test Suite
4.9 Included errors - .ER
4.9.1 Description
.ER [Bad]Access grab
.ER [Bad]Access colormap-free
.ER [Bad]Access colormap-store
.ER [Bad]Access acl
.ER [Bad]Access select
.ER [Bad]Alloc
.ER [Bad]Atom [val1] [val2] ... †
.ER [Bad]Color
.ER [Bad]Cursor [val1] [val2] ... †
.ER [Bad]Drawable [val1] [val2] ... †
.ER [Bad]Font bad-font
.ER [Bad]Font bad-fontable
.ER [Bad]GC
.ER [Bad]Match inputonly
.ER [Bad]Match gc-drawable-depth
.ER [Bad]Match gc-drawable-screen
.ER [Bad]Match wininputonly
.ER [Bad]Name font
.ER [Bad]Name colour
.ER [Bad]Pixmap [val1] [val2] ... †
.ER [Bad]Value <arg> [mask] <val1> [val2] ... ‡
.ER [Bad]Window [val1] [val2] ... †
† - these arguments are optional. ‡ - the <arg> and at least <val1> argument must be supplied.The mask argument, and additional arguments, are optional. Note - the Bad prefix is in each case optional. This keyword causes mc to insert into the C source file thetext of an assertion, and in some cases default strategy anddefault test code to test for the generation of a particularX Protocol error by an Xlib function. In some cases there is no strategy and code in the includedfile, because only the assertion is common - the strategyand code sections are specific to each test purpose, andmust be provided immediately after the .ER keyword. The default strategy and code sections (if included) may beoverridden by sections in the dot-m file immediately afterthe .ER keyword. Note that this keyword does not insert the >>ASSERTIONkeyword - this must appear on the line before .ER isinvoked. Thus the keyword does not include the entireassertion section. The assertion text, strategy and test code are included from
____________________
1.
Obtainable from Publication Sales, IEEE Service Center,
P.O. Box 1331, 445 Hoes Lane, Piscataway, NJ 08854-1331,
(201) 981-0060
- 9 -
Programmers Guide for the X Test Suite
files in the directory
$TET_ROOT/xtest/lib/error.
The names of these files, and the assertion text in each
file, is shown in appendix C.
4.10 Set options - >>SET
4.10.1 Description
>>SET startup <func_startup>
>>SET cleanup <func_cleanup>
>>SET tpstartup <func_tpstartup>
>>SET tpcleanup <func_tpcleanup>
>>SET need-gc-flush
>>SET fail-return
>>SET fail-no-return
>>SET return-value <return_value>
>>SET no-error-status-check
>>SET macro [ <macroname> ]
>>SET begin-function
>>SET end-function
These options control how the mc utility converts the dot-mfile into a C source file. Except where specifically stated, they may appear anywherein the dot-m file and apply from that point on, unless resetby a further >>SET keyword with the same first argument. 4.10.2 Arguments startup The name of the function called before all testpurposes is to be set to func_startup (ratherthan the default, startup()). cleanup The name of the function called after all testpurposes is to be set to func_cleanup (ratherthan the default, cleanup()). tpstartup The name of the function called before each testpurpose is to be set to func_tpstartup (ratherthan the default, tpstartup()). tpcleanup The name of the function called after each testpurpose is to be set to func_tpcleanup (ratherthan the default, tpcleanup()). need-gc-flush When the XCALL macro is expanded, code to callthe X Test Suite library functiongcflush(display, gc) will be inserted after thecode to call the function under test. fail-return When the XCALL macro is expanded, code to endthe test purpose will be inserted where an erroris reported (the default is to continue after anerror is reported).
- 10 -
Programmers Guide for the X Test Suite
fail-no-return
When the XCALL macro is expanded, no code to end the test
purpose will be inserted where an error is reported (this
reverses the effect earlier using >>SET
fail-return).
return-value
When the XCALL macro is expanded, and the Xlib function call
has return type Status, and the return value of
XCALL is not saved for testing in the calling code, code
will be inserted to report an error if the function under
test does not return <return_value>. (By default, when
the Xlib function call has return type Status, an
error is reported for assertions with test-type Good if the
return value is zero, and for assertions with test-type Bad
if the return value is non-zero).
no-error-status-check
When the XCALL macro is expanded, the default code to check
for X Protocol errors will not be inserted. The test purpose
can perform alternative checking after invoking XCALL. This
setting only applies up to the end of the current
section.
macro
There is a macro in an X Window System header file for which
a test set source file will be produced which uses identical
test purposes to the function under test. This is used to
automatically generate test purposes for the Display and
Screen information macros, which are identical to those for
the corresponding Xlib functions.
The macro name is set to <macroname> - the
default is the function argument in the
>>TITLE keyword, with the leading letter
‘X’ removed.
This option must be specified before the title
section of the dot-m file.
Note - this option may not be used for macros which have no
arguments.
begin-function
The name of an additional function called before each test
purpose, after tpstartup() and global function arguments are
initialised.
end-function
The name of an additional function called after each test
purpose, before tpcleanup().
4.11 Comment lines - >>#
4.11.1 Description
>># <Comment text>
This keyword specifies a one line comment. These are not
- 11 -
Programmers Guide for the X Test Suite
intended to replace code comments in the code
section - in fact the mc utility does not copy
dot-m format comments to the C source file. Comments in the
dot-m file are used for higher level comments rather than
detailed comments. For example, comments are used to record
the history of the development of the dot-m file, to
preserve previous versions of assertions where necessary,
and to draw attention to unresolved problem
areas.
- 12 -
Programmers Guide for the X Test Suite
5. Source file formats
This section describes the output from various utilities
which may be used to format the contents of a dot-m
file.
The code-maker utility mc builds C source files
from a dot-m file. Appendix D gives a usage summary.
The Makefile utility mmkf builds Makefiles from a
dot-m file. Appendix E gives a usage summary.
The assertion utility ma produces a list of
assertions from a dot-m file. Appendix F gives a usage
summary.
Instructions for building and installing the mc
utility are given in the "User Guide". When the
utility mc is built and installed, the utilities
mmkf and ma, which are links to the same
program mc differing only in name, are
automatically installed as well.
5.1 C files for standalone executable -
Test.c
The command
mc -o Test.c stclpmsk.m
produces a C file named Test.c. This may be compiled toproduce a standalone executable file named Test.Instructions on building and executing the tests are givenin the "User Guide". The Test.c files are not provided aspart of this release, but are built automatically whenbuilding the X Test Suite. The C file contains all of the interface code required toinvoke the test purposes from the TET and a description ofthe assertion being tested is placed as a comment above thecode for each test purpose to make it easy to understandwhat is being tested. The remaining parts of this section describe the format ofthe Test.c files in more detail. The descriptions are in theorder in which the text is inserted into the Test.c file. Some parts of the Test.c file are constructed by copying intemplate files specifically written to work with mc. Thesefiles are all located in the directory$TET_ROOT/xtest/lib/mc. 5.1.1 Copyright header A copyright header is inserted as a C source comment block.This will contain lines showing the SCCS versions of thedot-m file and any included files. 5.1.2 SYNOPSIS section A synopsis defining the arguments of the function beingtested is inserted as a C source comment block. This isconstructed from the lines following the >>TITLE keyword inthe dot-m file. It includes the data type returned and anyarguments to the function. The synopsis section is omitted if there are no linesfollowing the the >>TITLE keyword. For example:
- 13 -
Programmers Guide for the X Test Suite
/*
* SYNOPSIS:
* void
* XSetClipMask(display, gc, pixmap)
* Display *display;
* GC gc;
* Pixmap pixmap;
*/
5.1.3 Include files For the Xlib tests, when the section argument to the >>TITLEkeyword is other than XPROTO, the contents of the filemcinclude.mc are then included. In this release the contents of this file are as follows:
#include <stdlib.h>
#include "xtest.h"
#include "Xlib.h"
#include "Xutil.h"
#include "Xresource.h"
#include "tet_api.h"
#include "xtestlib.h"
#include "pixval.h"
For the X Protocol tests, when the section argument to the>>TITLE keyword is XPROTO the contents of the filemcxpinc.mc are then included. In this release the contents of this file are as follows:
#include <stdlib.h>
#include "xtest.h"
#include "tet_api.h"
5.1.4 External variables For the Xlib tests, when the section argument to the >>TITLEkeyword is other than XPROTO, the contents of the filemcextern.mc are then included. In this release the contents of this file are as follows:
- 14 -
Programmers Guide for the X Test Suite
extern Display *Dsp;
extern Window Win;
extern Window ErrdefWindow;
extern Drawable ErrdefDrawable;
extern GC ErrdefGC;
extern Colormap ErrdefColormap;
extern Pixmap ErrdefPixmap;
extern Atom ErrdefAtom;
extern Cursor ErrdefCursor;
extern Font ErrdefFont;
The external variables are defined in the file startup.c or(when linking a program executed via tet_exec()) in the fileex_startup.c. For the X Protocol tests, when the section argument to the>>TITLE keyword is XPROTO the contents of the filemcxpext.mc are then included. In this release this file is empty. 5.1.5 Test set symbol and name A symbol is defined indicating the function under test. For example:
#define T_XSetClipMask 1
You may use this in a #ifdef control line, to distinguishspecial cases for particular functions in code sections of adot-m file included via the >>INCLUDE keyword. The global variable TestName is initialised to the name ofthe function under test, which is the function argumentgiven to the >>TITLE keyword. For example:
char *TestName = "XSetClipMask";
You can use this within a code section of a dot-m file toobtain the name of the function under test. 5.1.6 Definitions for arguments Symbols are defined to correspond with any arguments to thefunction specified in the >>TITLE keyword. These correspond to the lines following the >>TITLE keywordin the dot-m file. For example:
- 15 -
Programmers Guide for the X Test Suite
/*
* Defines for different argument types
*/
#define A_DISPLAY display
#define A_GC gc
#define A_PIXMAP pixmap
#define A_DRAWABLE pixmap
These are used by the code in various included files, tosubstitute a symbol representing a particular argument typewith the actual variable used as the argument by thefunction under test. 5.1.7 Static variables Static variables are defined to correspond with anyarguments to the function specified in the >>TITLE keyword. These correspond to the lines following the >>TITLE keywordin the dot-m file. For example:
/*
* Arguments to the XSetClipMask function
*/
static Display *display;
static GC gc;
static Pixmap pixmap;
These are the arguments that will be passed to the functionunder test when the XCALL macro is expanded. You can initialise these in a code section of a dot-m fileas required prior to invoking the macro XCALL. These variables will be initialised at the start of eachtest purpose using the function setargs() described below inthe section entitled "Initialising arguments". 5.1.8 Test purpose number You can use this within a code section of a dot-m file toobtain the number of the current test purpose.
int tet_thistest;
5.1.9 Initialising arguments A function setargs() is defined to initialise the argumentsto the function under test. Code to call this function is inserted at the start of eachtest purpose before the code you put in the code section. The arguments are initialised to have the value of theexpression you specified in the title section. This does nothave to be a constant expression - for example, it my be a
- 16 -
Programmers Guide for the X Test Suite
return value of a function in a library or extern
section. By default arguments are initialised to zero
values.
For example:
/*
* Called at the beginning of each test purpose to reset the
* arguments to their initial values
*/
static void
setargs()
{
display = Dsp;
gc = 0;
pixmap = 0;
}
5.1.10 Initialising arguments when test-type is Bad A function seterrdef() is defined to initialise some of thearguments to the function under test to values which aresuitable for conducting the included error tests. This is called in some of the included error tests toinitialise the arguments to known good values. For example:
/*
* Set arguments to default values for error tests
*/
static void
seterrdef()
{
gc = ErrdefGC;
pixmap = ErrdefPixmap;
}
5.1.11 Code sections The code sections in the dot-m file are converted into asequence of functions named tnnn(), where nnn is a threedigit number which is filled with leading zeros ifnecessary, the first code section being named t001(). Thisis known as a "test function". Each of the test functions is preceded by the correspondingassertion for the test purpose, in a C source code comment,labelled with the test purpose number. Code to call the library function tpstartup() is inserted atthe start of the test function, immediately after anyautomatic variables declared in your code section. Thisfunction performs initialisation required for each testpurpose, including setting error handlers to trap unexpectederrors. Code to call the automatically generated test-set specificfunction setargs() is then inserted. This function is
- 17 -
Programmers Guide for the X Test Suite
described further in the previous section
entitled "Initialising arguments".
The contents of your code section are then inserted.
The macro XCALL in a dot-m file is expanded to call the
function under test with arguments corresponding to the
lines following the >>TITLE keyword.
For example:
XCALL;
is by default expanded to
startcall(display);
if (isdeleted())
return;
XSetClipMask(display, gc, pixmap);
endcall(display);
if (geterr() != Success) {
report("Got %s, Expecting Success", errorname(geterr()));
FAIL;
}
The stages in this expansion are as follows: 1. The library function startcall() is called to checkfor any outstanding unexpected X protocol errors,which might have been generated, for example, duringthe setup part of the test. A call to XSync() is madeto achieve this. 2. The library function startcall() installs a test errorhandler in place of the unexpected X protocol errorhandler. 3. The library function isdeleted() returns True if thetest purpose has already issued a result codeUNRESOLVED due to an earlier call to delete(). (Thismust be done after calling startcall() in case XSync()flushed an unexpected X protocol error.) 4. The function under test is then called with thearguments listed in the title section. 5. The library function endcall() is called to check forany X protocol errors caused by the function undertest. A call to XSync() is made to achieve this. 6. The library function endcall() installs the unexpectedX protocol error handler. 7. The test error handler saves the number of the mostrecent X protocol error. It is accessed by callingthe function geterr() and the value is checked. Thevalue is expected to be Success by default, orBadThing if the code section was introduced in thedot-m file by
>>CODE BadThing
- 18 -
Programmers Guide for the X Test Suite
If it is desirable to skip checking the error
status at this point, the option
>>SET no-error-status-check
may be inserted in the dot-m file in the current codesection before the XCALL. This setting only appliesup to the end of the current section. 5.1.12 TET initialisation code The mc utility adds a reference to the function into anarray of functions which can be invoked via the TET API. For example:
struct tet_testlist tet_testlist[] = {
t001, 1,
t002, 2,
t003, 3,
NULL, 0
};
Code to calculate the number of test purposes in the testset is inserted as follows:
int ntests = sizeof(tet_testlist)/sizeof(struct tet_testlist)-1;
Finally, the names of the startup and cleanup functions areused to initialise variables used by the TET API. Thesefunctions are called before the first test purpose and afterthe last test purpose. The functions called can beoverridden using the options >>SET startup and >>SETcleanup. The default library functions perform initialisationincluding reading the TET configuration variables andopening a default client. Should you override the default startup and cleanupfunctions, you are recommended to call startup() as thefirst line of your startup function and cleanup() as thelast line of your cleanup function.
void (*tet_startup)() = startup;
void (*tet_cleanup)() = cleanup;
5.2 C files for standalone executable in macro tests -MTest.c When the dot-m file contains the line
- 19 -
Programmers Guide for the X Test Suite
>>SET macro
the command
mc -m -o MTest.c scrncnt.m
produces a C file named MTest.c. This may be compiled toproduce a standalone executable file named MTest.Instructions on building and executing the tests are givenin the "User Guide". The MTest.c files are not provided aspart of this release, but are built automatically whenbuilding the X Test Suite. The file MTest.c is identical to the file Test.c except thata macro (which is expected to be made visible by includingthe file Xlib.h) is tested instead of the Xlib functionnamed in the title section of the dot-m file. The macro name is set to the <macroname> argument of the>>SET macro option - if there is no >>SET macro option inthe file, or no argument specified, the default is thefunction argument in the >>TITLE keyword, with the leadingletter ‘X’ removed. 5.3 C files for linked executable - link.c The command
mc -l -o link.c stclpmsk.m
produces a C file named link.c. This is identical to theTest.c file with the exception of the initialisation codewhich enables the source code to be compiled and linked intoa space-saving executable file. This is an executable filewhich may invoke any of the test purposes in the variouslink.c files, thereby reducing the number of executablefiles required, and saving space. The link.c files are notprovided as part of this release, but are builtautomatically when building the X Test Suite. The remaining parts of this section describe the differencesin format of the link.c and Test.c files. The differences are associated with the TET initialisationcode being in a separate source file named linktbl.c, ratherthan in the test set source file. A linktbl.c file is provided for each section of the X TestSuite in each subdirectory of $TET_ROOT/xtest/tset. Thisfile contains a pointer to an array of linkinfo structures,one for each test set in the section. Each linkinfostructure contains the following items: name a unique name for that test set (thename of the test set directory). testname the actual Xlib function tested by thetest set.
- 20 -
Programmers Guide for the X Test Suite
ntests the number of test
purposes in the test set.
testlist a pointer to the array of test functions
constructed for that test set from the contents of the
link.c file.
localstartup a pointer to the startup function specific
to that test set.
localcleanup a pointer to the cleanup function specific
to that test set.
Later in this section there are example values of these
structure members.
When the space-saving executable is executed, the TET
initialisation code in the library function
linkstart.c determines which test set is required.
This is done by matching argv[0] with a
name element in the array of linkinfo
structures. The test functions specified by the
corresponding testlist element of the
linkinfo structure are then executed, preceded and
followed by the corresponding startup and cleanup function
respectively.
5.3.1 Test set symbol and name
The global variable TestName is made
static.
static char *TestName = "XSetClipMask";
5.3.2 Test purpose number This is defined in linktbl.c, and is made available via thefollowing code:
extern int tet_thistest;
5.3.3 TET initialisation code The global variable tet_testlist is made static. For example:
static struct tet_testlist tet_testlist[] = {
t001, 1,
t002, 2,
t003, 3,
NULL, 0
};
The global variable ntests is made static.
static int ntests = sizeof(tet_testlist)/sizeof(struct tet_testlist)-1;
The linkinfo structure specific to this test set is defined.
- 21 -
Programmers Guide for the X Test Suite
For example:
struct linkinfo EXStClpMsk = {
"stclpmsk",
"XSetClipMask",
&ntests,
tet_testlist,
0,
0,
};
The TET variables for the startup and cleanup functions aredefined in linktbl.c, and are made available via thefollowing code:
extern void (*tet_startup)();
extern void (*tet_cleanup)();
5.4 C files for linked executable in macro tests - mlink.c When the dot-m file contains the line
>>SET macro
the command
mc -m -l -o mlink.c scrncnt.m
produces a C file named mlink.c. The mlink.c files are notprovided as part of this release, but are builtautomatically when building the X Test Suite. The file mlink.c is identical to the file link.c, exceptthat a macro (which is expected to be made visible byincluding the file Xlib.h) is tested instead of the Xlibfunction named in the title section of the dot-m file. The macro name is set to the <macroname> argument of the>>SET macro option - if there is no >>SET macro option inthe file, or no argument specified, the default is thefunction argument in the >>TITLE keyword, with the leadingletter ‘X’ removed. 5.5 Makefile The command
mmkf -o Makefile scrncnt.m
produces a Makefile which can be used to build all the C
- 22 -
Programmers Guide for the X Test Suite
source files described in the previous sections
and to build the test executables from the C files.
Further instructions appear in the "User Guide" in
the section entitled "Building, executing and reporting
tests without using the TET".
The Makefiles produced by mc are portable in that
they use symbolic names to describe commands and parameters
which may vary from system to system. The values of these
symbolic names are all obtained by a utility pmake
from the build configuration file, which is described in the
"User Guide" in the section entitled
"Configuring the X Test Suite".
The targets in the Makefile which can be invoked by
pmake are as follows:
pmake Test
Builds standalone executable version of the test
set.
pmake Test.c
Builds Test.c using mc with the format
described in the earlier section entitled "C files for
standalone executable - Test.c".
pmake MTest
Builds standalone executable version of the test set to test
the macro version of the function.
pmake MTest.c
Builds MTest.c using mc with the format
described in the earlier section entitled "C files for
standalone executable - MTest.c".
pmake linkexec
Builds the object files and links which can be used to
produce a linked executable file. These targets are used
when building space-saving executables as described in the
"User Guide".
pmake link.c
Builds link.c using mc with the format
described in the earlier section entitled "C files for
linked executable - link.c".
pmake mlink.c
Builds mlink.c using mc with the format
described in the earlier section entitled "C files for
linked executable - mlink.c".
pmake clean
This removes object files and temporary files from the test
set directory.
pmake clobber
This removes object files and temporary files from the test
set directory and additionally removes all the source files
which mc can remake.
The remaining parts of this section describe the format of
the Makefiles in more detail.
Refer to the section entitled "Make section -
>>MAKE" for examples of how the variables
AUXFILES and AUXCLEAN may be
set.
5.5.1 Copyright header
A copyright header is inserted as a comment block. This will
contain lines showing the SCCS versions of the dot-m
file
- 23 -
Programmers Guide for the X Test Suite
and any included files.
5.5.2 Make variables
A series of make variables are initialised to
represent the names of the source, object and executable
files.
SOURCES=scrncnt.m
CFILES=Test.c
OFILES=Test.o
MOFILES=MTest.o
LOFILES=link.o mlink.o
LINKOBJ=scrncnt.o
LINKEXEC=scrncnt
5.5.3 Targets for X Protocol tests For the X Protocol tests, when the section argument to the>>TITLE keyword is XPROTO the contents of the filemmxpinit.mc are then included. This file initialises various make variables to specificvalues for the X Protocol tests. In this release the contents of this file are as follows:
#
# X Protocol tests.
#
# CFLAGS - Compilation flags specific to the X Protocol tests.
#
CFLAGS=$(XP_CFLAGS)
SYSLIBS=$(XP_SYSLIBS)
LIBS=$(XP_LIBS)
# LINTFLAGS - Flags for lint specific to the X Protocol tests.
#
LINTFLAGS=$(XP_LINTFLAGS)
LINTLIBS=$(XP_LINTLIBS)
5.5.4 Targets for standalone executable - Test The contents of the file mmsa.mc are included. These are thetargets to create the standalone executable file Test.
- 24 -
Programmers Guide for the X Test Suite
#
# Build a standalone version of the test case.
#
Test: $(OFILES) $(LIBS) $(TCM) $(AUXFILES)
$(CC) $(LDFLAGS) -o $@ $(OFILES) $(TCM) $(LIBLOCAL) $(LIBS) $(SYSLIBS)
Test.c: $(SOURCES)
$(CODEMAKER) -o Test.c $(SOURCES)
5.5.5 Targets for standalone executable - MTest If the dot-m file contains the >>SET macro option, thecontents of the file mmmsa.mc are included. These are thetargets to create the standalone executable file MTest forthe macro version of the specified Xlib function.
#
# Build a standalone version of the test case using the macro version
# of the function.
#
MTest: $(MOFILES) $(LIBS) $(TCM) $(AUXFILES)
$(CC) $(LDFLAGS) -o $@ $(MOFILES) $(TCM) $(LIBLOCAL) $(LIBS) $(SYSLIBS)
MTest.c: $(SOURCES)
$(CODEMAKER) -m -o MTest.c $(SOURCES)
5.5.6 Targets for linked executable The contents of the file mmlink.mc are included. These arethe targets to create object files and links which can beused to produce a linked executable file. These targets areused when building space-saving executables as described inthe "User Guide".
- 25 -
Programmers Guide for the X Test Suite
#
# A version of the test that can be combined together with
# all the other tests to make one executable. This will save a
# fair amount of disk space especially if the system does not
# have shared libraries. Different names are used so that
# there is no possibility of confusion.
#
link.c: $(SOURCES)
$(CODEMAKER) -l -o link.c $(SOURCES)
# Link the objects into one large object.
#
$(LINKOBJ): $(LOFILES)
$(LD) $(LINKOBJOPTS) $(LOFILES) -o $(LINKOBJ)
# Link the object file into the parent directory.
#
../$(LINKOBJ): $(LINKOBJ)
$(RM) ../$(LINKOBJ)
$(LN) $(LINKOBJ) ..
# Make a link to the combined executable.
#
$(LINKEXEC): ../Tests
$(RM) $(LINKEXEC)
$(LN) ../Tests $(LINKEXEC)
../Tests: ../$(LINKOBJ)
linkexec:: $(LINKEXEC) $(AUXFILES) ;
5.5.7 Targets for linked executable - macro version If the dot-m file contains the >>SET macro option, thecontents of the file mmmlink.mc are included. These are thetargets to create object files and links for the macroversion of the specified Xlib function which can be used toproduce a linked executable file. These targets are usedwhen building space-saving executables as described in the"User Guide".
- 26 -
Programmers Guide for the X Test Suite
# A version of the test that can be combined with all the other tests for
# the macro version of the function.
#
mlink.c: $(SOURCES)
$(CODEMAKER) -m -l -o mlink.c $(SOURCES)
linkexec:: m$(LINKEXEC) $(AUXFILES) ;
m$(LINKEXEC): ../Tests
$(RM) m$(LINKEXEC)
$(LN) ../Tests m$(LINKEXEC)
5.5.8 Targets for libraries For the Xlib tests, when the section argument to the >>TITLEkeyword is other than XPROTO, the contents of the filemmlib.mc are then included. In this release the contents of this file are as follows:
#
# This part of the makefile checks for the existance of the libraries
# and creates them if necessary.
#
# The xtestlib is made if it doesn’t exist
#
$(XTESTLIB):
cd $(XTESTROOT)/src/lib; $(TET_BUILD_TOOL) install
# The fontlib is made if it doesn’t exist
#
$(XTESTFONTLIB):
cd $(XTESTROOT)/fonts; $(TET_BUILD_TOOL) install
For the X Protocol tests, when the section argument to the>>TITLE keyword is XPROTO the contents of the filemmxplib.mc are then included. This file is identical tommlib.mc except for the following additional lines:
# The X Protocol test library is made if it doesn’t exist
#
$(XSTLIB):
cd $(XTESTROOT)/src/libproto; $(TET_BUILD_TOOL) install
- 27 -
Programmers Guide for the X Test Suite
5.5.9 Targets for cleaning and linting
The contents of the file mmmisc.mc are then
included.
This file includes a clean target to remove object
files and temporary files, and a clobber target
which additionally removes all the source files which
mc can remake.
There is also a LINT target which enables the C
source files to be checked against lint libraries specified
in the build configuration file.
#
# Miscellaneous housekeeping functions.
#
# clean up object and junk files.
#
clean:
$(RM) Test $(OFILES) $(LOFILES) $(LINKOBJ) $(LINKEXEC) core\
MTest m$(LINKEXEC) $(MOFILES) CONFIG Makefile.bak $(AUXCLEAN)
# clobber - clean up and remove remakable sources.
#
clobber: clean
$(RM) MTest.c Test.c mlink.c link.c Makefile
# Lint makerules
#
lint: $(CFILES)
$(LINT) $(LINTFLAGS) $(CFILES) $(LINTTCM) $(LINTLIBS)
LINT:lint
5.5.10 Targets for building known good image files The contents of the file mmpgen.mc are then included. These include targets which enable the test set to be builtso that it generates known good image files. These are not intended to be used outside the developmentenvironment at UniSoft.
- 28 -
Programmers Guide for the X Test Suite
#
# Pixel generation makerules for generating the reference
# known good image files.
#
PVOFILES=pvtest.o
pvgen: $(PVOFILES) $(PVLIBS) $(TCM)
$(CC) $(LDFLAGS) -o $@ $(PVOFILES) $(TCM) \
$(PVLIBS) $(SYSLIBS) $(SYSMATHLIB)
pvtest.o: pvtest.c
cc -c -DGENERATE_PIXMAPS $(CFLAGS) pvtest.c
pvtest.c: Test.c
$(RM) pvtest.c; \
$(LN) Test.c pvtest.c
5.5.11 Targets for included files Rules are included to specify the dependency of the C sourcefiles on any included files. For example:
Test.c link.c: $(XTESTLIBDIR)/error/EAll.mc
Test.c link.c: $(XTESTLIBDIR)/error/EGC.mc
Test.c link.c: $(XTESTLIBDIR)/error/EPix.mc
5.6 Formatting assertions The command
ma -o stclpmsk.a -h -m stclpmsk.m
produces in the file stclpmsk.a a list of the assertionsfrom the assertion sections of the specified dot-m file.The assertions are output in nroff format. All macros usedin the assertion text can be obtained using the -h and -soptions as described below. The remaining parts of this section describe the outputformat in more detail. 5.6.1 Copyright header A copyright header is output as an nroff comment block.
- 29 -
Programmers Guide for the X Test Suite
This will contain lines showing the SCCS versions
of the dot-m file and any included files.
5.6.2 Macro definitions
If the -h option was specified, macros that are later used
in the assertion text will be output from the file
maheader.mc.
5.6.3 Title
The line
.TH <function> <section>
is output, where <function> and <section> are obtained fromthe title section of the dot-m file. The default macro definition for .TH in maheader.mc causesthe section and function name to be printed at the top ofeach page. 5.6.4 Assertions For each assertion section, the line
.TI <category> \" <function>-n
is output, where <category> is obtained from the secondargument of the >>ASSERTION keyword and <function> isobtained from the title section of the dot-m file, and n isthe number of the assertion in the test set. This is followed by the assertion text in which xname isconverted to <function>. For example:
.TI A \" XSetClipMask-1
A call to
.F XSetClipMask
sets the
.M clip_mask
component of the specified GC to the value of the
.A pixmap
argument.
The other macros used in the assertion text to control fontchanges are described in appendix B. The default macro definition for .TH in maheader.mc causesthe example assertion to be printed as follows:
Assertion XSetClipMask-1(A).
A call to XSetClipMask sets the clip_mask component of the
specified GC to the value of the pixmap argument.
- 30 -
Programmers Guide for the X Test Suite
- 31 -
Programmers Guide for the X Test Suite
6. Source file structure
This section describes the C source coding style and
conventions which which have been used in the development of
the revised X Test Suite. These conventions apply to the
structure of the code sections of the dot-m files, whose
overall structure is defined in previous sections of the
Programmers Guide. In some cases (particularly in the
structure of the X Protocol tests) the style and conventions
have been deleloped from the earlier T7 X Test Suite.
You are advised to study the contents of this section before
attempting to modify or extend the X Test Suite. The
contents of this section will give you guidelines on how to
structure the test code so that it is easy to follow, gives
correct and reliable information when the tests are
executed, and is written as compactly as possible.
Libraries of common functions have been used and further
developed in the revised X Test Suite in order to keep the
source code in the test sets as compact as possible. The
rest of this section describes recommendations on how
particular library functions should be used. It does not
describe the contents of the libraries in detail. A complete
list of library contents is provided in the section entitled
"Source file libraries".
During the development of the Xlib tests, a library of
support functions has been developed. This library includes
functions for performing common operations required when
testing the X Window System, as well as performing common
reporting operations. This library includes a small number
of functions developed for the Xlib tests within the T7 X
Test Suite. This library is known as the "X test suite
library" in this document, and the source of the
library is in the directory
$TET_ROOT/xtest/src/lib.
Calls to any function in this library may be made by any
test set in the X Test Suite.
6.1 Structure of the Xlib tests
This section describes the structure of the code sections of
the Xlib tests.
The Xlib tests are the tests for sections 2 to 10 of the
X11R4 Xlib specifications. They are stored in subdirectories
of the directories CH02 to CH10 (which are
to be found in the directory $TET_ROOT/xtest/tset).
There is a subdirectory for each Xlib function containing a
dot-m file which includes all the test purposes provided for
that Xlib function. The naming scheme which is used for
these directories is described in appendix B of the
"User Guide".
6.1.1 Result code macros
It is good practice where possible to structure the test so
that only one test result code is assigned before the code
section returns or ends.
The significance of the various test result codes that may
be assigned are described more fully in appendix D of the
"User Guide".
The following macros may be used to assign the test
result
- 32 -
Programmers Guide for the X Test Suite
code. These macros call the function
tet_result() which is part of the TET API.
PASS
This assigns test result code PASS.
FAIL
This assigns test result code FAIL.
UNRESOLVED
This assigns test result code UNRESOLVED.
UNSUPPORTED
This assigns test result code UNSUPPORTED.
UNTESTED
This assigns test result code UNTESTED.
NOTINUSE
This assigns test result code NOTINUSE.
WARNING
This assigns test result code WARNING.
FIP
This assigns test result code FIP.
Note that there are two other test result codes which may
not be assigned directly within a test purpose.
The result code UNINITIATED will be assigned to a test
purpose from within the TET when the function
tet_delete() has been called in an earlier test
purpose or startup function. This is useful to prevent
initiation of later test purposes when it is not possible to
continue executing test purposes in the test set.
The result code NORESULT will be assigned to a test purpose
from within the TET if the test purpose is initiated but no
result code has been output by the time control returns from
the test purpose to the TET.
The FAIL macro also increments a failure counter which is
used to prevent a result code being assigned in a later call
to CHECKPASS (see below).
6.1.2 Result code functions
There are a series of convenience functions which output a
particular test result code preceded by a test information
message of type REPORT. (See "Outputting test
information messages", below).
In each case the arguments are exactly like those for
printf(3).
These are as follows:
untested()
This function may be used for an extended assertion to
output the test result code UNTESTED, preceded by a
message.
unsupported()
This function may be used for a conditional assertion to
output the test result code UNSUPPORTED, preceded by a
message.
notinuse()
This function may be used to output the test result code
NOTINUSE, preceded by a message.
delete()
This function may be used to output the test result code
UNRESOLVED, preceded by a message.
- 33 -
Programmers Guide for the X Test Suite
6.1.3 Assigning result codes
The code should be structured such that a PASS result code
is only assigned if there is no doubt that the assertion
being tested has been determined to be positively true on
the system being tested. Absence of failure should not be
taken as proof of success. For this reason, there should if
possible be just one place where a PASS result may be
assigned, whilst there may be many code paths which report
other result codes.
The result code FAIL should not be called until the function
under test has been called.
During execution of the test purpose, it may not be possible
to setup the conditions for the assertion to be conclusively
tested. In this case the result code UNRESOLVED should be
assigned rather than FAIL.
For example:
>>CODE
if (setup()) {
delete("setup() failed; the test could not be completed");
return;
}
ret = XCALL;
if (ret == 0)
PASS;
else
fAIL;
6.1.4 Assigning result codes for extended assertions Extended assertions are described in more detail as part ofthe >>ASSERTION keyword. In some cases partial testing may be done for extendedassertions. In this case, the result code would be FAIL ifan error is detected, or UNTESTED if no failure is detectedbut the assertion is still not fully tested. For example:
>>CODE
ret = XCALL;
if (ret == 0)
PASS;
else
- 34 -
Programmers Guide for the X Test Suite
untested("The assertion could not be completely tested"); |
Programmers Guide for the X Test Suite
>>CODE
n_ret = -1;
ret = XCALL;
if (ret == 0)
CHECK;
else
FAIL;
if (n_ret == expected_number)
CHECK;
else
FAIL;
CHECKPASS(2);
In the case of extended assertions, the macro CHECKUNTESTEDmay be called, which is identical to CHECKPASS, except thatthe final result code assigned will be UNTESTED. 6.1.7 Outputting test information messages Test information messages are normally output to describethe reason for any test result codes which are other thanPASS, and for other purposes, as described in this section. Appendix D of the "User Guide" describes the four differentcategories of test information messages which may appear inthe TET journal file. This section describes how thesemessages are output from the test purpose. The functions described in this section call the functiontet_infoline() which is part of the TET API. REPORT A test information message with type REPORT isused to report the reason for any test resultcode which is other than PASS. A warning messageis printed by the report writer rpt if a testinformation message of type REPORT is given in atest purpose which produced a test result codePASS. This is output using the function report(),which takes arguments exactly like those forprintf(3). CHECK A test information message with type CHECKshould not be output directly - this should onlybe done via the CHECK macro. TRACE A test information message with type TRACE isused to describe the state of the test beingexecuted.
- 36 -
Programmers Guide for the X Test Suite
This is not output to the TET journal file if the
execution configuration parameter XT_OPTION_NO_TRACE is set
to Yes.
This is output using the function trace(), which
takes arguments exactly like those for printf(3).
DEBUG
A test information message with type DEBUG is a debug
message inserted during the development of the test.
This is only output to the TET journal file if the value of
the execution configuration parameter XT_DEBUG is greater
than or equal to the level of the debug message.
This is output using the function debug(), which
takes arguments exactly like those for printf(3),
except that the printf(3) arguments are preceded by a
single argument which is the debug level. The debug level
should be between 1 and 3.
For example:
>>CODE
debug(1, "about to call %s", TestName);
ret = XCALL;
if (ret == 0)
trace("%s returned %d", TestName, ret);
PASS;
} else {
report("%s returned %d instead of 0", TestName, ret);
FAIL;
}
6.1.8 Creating new test purposes You can create new test purposes within an existing dot-mfile using the guidelines in this section. It is expected that in doing this you will be primarilyaiming to produce new test purposes for a particular Xlibfunction. You should add the new test purpose to the dot-mfile containing the test purposes for that Xlib function. 6.1.8.1 Creating new sections in the dot-m file You are advised to create an assertion section and strategysection at the end of the file, using as a template one ofthe existing sections in the dot-m file. You should then create a code section commencing with the>>CODE keyword. Since there are many different styles ofXlib functions which may be tested, there are few additionalguidelines that can be given beyond those contained inearlier parts of this guide.
- 37 -
Programmers Guide for the X Test Suite
6.1.8.2 Creating test purposes which use pixmap
verification
If you have not done so yet, refer to the section entitled
"Examining image files" in the "User
Guide". This explains some background to the pixmap
verification scheme, and in particular how to view image
files produced when running the X Test Suite.
A number of test purposes supplied in the X Test Suite use a
scheme known as pixmap verification, to compare the image
produced by the X server with a known good image which is
stored in a known good image file.
All the required known good image files for the test
programs in the X Test Suite (as supplied) have been created
in advance. The known good image files for each test program
are supplied in the X Test Suite in the test set directory
in which the dot-m file is supplied. They are named
annn.dat, where nnn is the number of the
test purpose for which the known good image file was
generated.
The known good image files are generated as follows. The X
Test Suite is compiled with the additional compilation flag
-DGENERATE_PIXMAPS, and linked with a replacement Xlib which
determines analytically the expected X server display
contents at any point. At the points where pixmap
verification is going to be performed, the expected image is
instead written to a data file, which is the known good
image file.
It is not possible to generate further known good image
files in this way, because the replacement Xlib is not part
of the X Test Suite.
However, it is possible to write a server-specific image
file containing the contents of the X server display at
points where pixmap verification is going to be performed.
This may be useful for the purposes of validation and
regression testing against a known server. This may be done
by working through the following stages:
1. Create the test purpose with a call to the macro PIXCHECK
at the point where you want to validate the image displayed
by the X server. Note that the macro PIXCHECK calls the
macros CHECK or FAIL depending on whether the image
displayed by the X server matches that in the image file.
The code invoked by the macro PIXCHECK(display, drawable) is
as follows:
if (verifyimage(display, drawable, (struct area *)0))
CHECK;
else
FAIL;
The function verifyimage() is described in more detailin the section entitled "Source file libraries". 2. Build and execute the test without using the TCC,(refer to the "User Guide") and check that the newlycreated test purpose gives result code UNRESOLVED dueto the absence of a known good image file as follows:
- 38 -
Programmers Guide for the X Test Suite
pmake
pt
prp
3. Rerun the test, saving the image produced by the Xserver as follows:
pt -v XT_SAVE_SERVER_IMAGE=Yes
4. This should create a file named annn.sav, where nnn isthe name of the newly created test purpose. This is aserver-specific image file. Rename this file to thename used for known good image files as follows:
mv annn.sav annn.dat
5. Check that the process has worked by executing thetest without using the TCC, and enabling pixmapverification against the server-specific image file asfollows:
pt
prp
The newly created test purpose should give a resultcode of PASS. It is particularly important that new test purposes areadded at the end of the file if an earlier test purposecalls the macro PIXCHECK. This is because inserting a testpurpose before another test purpose will cause the latertest purpose to be renumbered. As well as causingunnecessary confusion in other ways, this will cause thelater test purpose to now look for the wrong known goodimage file. 6.2 Structure of the X Protocol tests This section describes the structure of the code sections ofthe X Protocol tests. The X Protocol tests are the touch tests for the X Protocol(version 11). They are stored in subdirectories of thedirectory XPROTO (which is to be found in the directory$TET_ROOT/xtest/tset). There is a subdirectory for each XProtocol request containing a dot-m file which includes allthe test purposes provided for that X Protocol request. Thenaming scheme which is used for these directories isdescribed in appendix B of the "User Guide". During the development of the X Protocol tests, extensiveuse has also been made of a library of support functions
- 39 -
Programmers Guide for the X Test Suite
developed in the earlier T7 X Test Suite. This
library is known as the "X Protocol library" in
this document, and the source of the library is in the
directory $TET_ROOT/xtest/src/libproto.
Calls to any function in this library may be made by any of
the X Protocol tests in the X Test Suite.
6.2.1 Structure of the code sections
In the T7 release of the X Test Suite, each of the X
Protocol tests consisted of a main() function which
called library functions to send an X Protocol request to
the X server, and checked for the correct response (reply,
error or nothing).
In the revised X Test Suite, the test code originally in the
main() function has been moved to a function called
tester() which is located in an >>EXTERN
section in each dot-m file, so that it can be called from
each test purpose as described below. The test function
tester() is in turn called from a library function
testfunc().
For example:
>>CODE
test_type = GOOD;
/* Call a library function to exercise the test code */
testfunc(tester);
By default, the library function testfunc() calls tester()for each byte orientation. The test function tester() iscalled in a sub-process via the TET API function tet_fork(),and returns the exit status of the test process totestfunc(). If required, the execution configuration parameterXT_DEBUG_BYTE_SEX may be set to NATIVE, REVERSE, MSB or LSBto call tester() just once with the required byteorientation. Each client has a test type, which is initialised when theclient is created. The test type determines whether XProtocol requests sent by the client are to be good requestsor invalid requests (expecting an X Protocol error to bereturned). The test type may be modified during the lifetimeof the client by invoking the macro Set_Test_Type(). Inmany tests, this is done by setting the test type to one ofthe following values before calling Send_Req(), then settingit to GOOD immediately afterwards for subsequent librarycalls: GOOD The request sent will be a known good X Protocolrequest (unless otherwise modified in tester()before calling Send_Req()). BAD_LENGTH The request sent will have length field less
- 40 -
Programmers Guide for the X Test Suite
than the minimum needed to contain the
request.
JUST_TOO_LONG
The request sent will have length field greater than the
minimum needed to contain the request (and, for requests
where the length is used to determine the number of fields
in the request, the length is also not the minimum length
plus a multiple of the field size).
TOO_LONG
The request sent will have a length field which is greater
than that accepted by the X server under test.
BAD_IDCHOICE1
The request sent will have a resource ID that is already in
use (it is the responsibility of the function
tester() to ensure the resource ID is in use before
calling Send_Req()).
BAD_IDCHOICE2
The request sent will have a resource ID that is out of
range (it is the responsibility of the function
tester() to ensure the resource ID is out of range
before calling Send_Req()).
BAD_VALUE
The request sent will have an invalid mask bit set (it is
the responsibility of the function tester() to
ensure the mask field contains an invalid bit before calling
Send_Req()).
OPEN_DISPLAY
A special value used only in the test for OpenDisplay for
testing the connection setup protocol.
SETUP
The initial test type of a client, which will cause errors
during test setup to produce result code
UNRESOLVED rather than FAIL
.
6.2.2 Outputting test information and result
code
Errors may be detected and reported both within the test
function tester() and within library functions.
When an error is detected, the function Log_Err()
should be called. This increments an error count and uses
report() to output a test information message of
type REPORT to the TET journal file.
If no error is detected, the function Log_Trace() may be
called to record that the expected response was received.
This uses trace() to output a test information
message of type TRACE to the TET journal file.
You can also use the function Log_Debug() to output more
detailed test information such as the contents of request,
reply and event structures. This uses debug() to
output a test information message of type DEBUG at level one
to the TET journal file.
The function Exit() should be called at any point
after an error has occurred, which will assign a test result
code FAIL and print the error count (or UNRESOLVED if the
error counter is zero). The exit status will be EXIT_FAIL in
this case.
- 41 -
Programmers Guide for the X Test Suite
If tester() performs all checks and the
results are correct, the function Exit_OK() should
be called. The exit status will be EXIT_SUCCESS in this
case.
A result code of PASS is only assigned to a test purpose in
the library function testfunc() if all calls to
tester() give exit status EXIT_SUCCESS. It should
not be assigned anywhere else.
6.2.3 Creating new test purposes
You can create new test purposes within an existing dot-m
file using the guidelines in this section.
It is expected that in doing this you will be primarily
aiming to produce new test purposes for a particular X
Protocol request. You should add the new test purpose to the
dot-m file containing the test purposes for that X Protocol
request.
6.2.3.1 Creating new sections in the dot-m
file
You are advised to create an assertion section and strategy
section at the end of the file, using as a template one of
the existing sections in the dot-m file.
You should then create a code section which passes a
function my_test() you are about to create to the
library function testfunc().
For example:
>>CODE
test_type = GOOD;
/* Call a library function to exercise the test code */
testfunc(my_test);
6.2.3.2 Creating a new test function You should create a function my_test() in an >>EXTERNsection in the dot-m file using the guidelines in thissection. A client is a connection to the X server under test. Each XProtocol request is sent from a particular client. You cancreate a client numbered client using Create_Client(client).Normally a single client is created, but it is possible tocreate more than one client. This will be necessary whentesting the effect on the server of multiple clients. The client data structure Xst_clients is used to store theinformation about each client you have created. Thisincludes resource ID’s and a display structure which isfilled in when the client is created. The client datastructure is documented in more detail in the header file inwhich it is defined ($TET_ROOT/xtest/include/Xstlib.h). Next you will need to create a request structure. Thefunction Make_Req(client, req_type) should be called tocreate a request of a specified type req_type for a
- 42 -
Programmers Guide for the X Test Suite
specified client client and return a
pointer to the request structure. The request structure will
be filled in with defaults which may be suitable for the
test purpose you are creating. The file MakeReq.c
in the X Protocol library fills in the default values.
Should you want to change the defaults you can do this at
any point between creating the request structure and sending
it to the X server. It may be modified by accessing the
structure members. The format of the request structures is
exactly as defined in your X Protocol header file (normally
/usr/include/X11/Xproto.h). You can alter value
list items using the following functions:
Add_Masked_Value()
Del_Masked_Value()
Clear_Masked_Value()
Add_Counted_Value()
Clear_Counted_Value()
Add_Counted_Bytes()
When you have the request structure you wish to pass to the
X server, call the function Send_Req(client). This
sends the request req from client client
to the X server, and handles byte swapping and request
packing as necessary. If you wish the X Protocol library to
further modify the request structure to send an invalid
protocol request, set the test type of the client before
calling Send_Req(client) using the macro
Set_Test_Type(client). The possible test types are
listed in the earlier section entitled "Structure of
the code sections".
To check that the X server has reacted correctly to the
request sent, you will need to call the function
Expect(). For convenience, a number of macros and
functions have been created to call Expect()
depending on the outcome you are expecting. These are as
follows:
Expect_Event(client, event_type)
This expects an event of type event_type to be sent
back from the X server to client client.
Expect_Reply(client, req_type)
This expects a reply to the X Protocol request of type
req_type to be sent back from the X server to
client client.
Expect_Error(client, error_type)
This expects an error of type error_type to be sent
back from the X server to client client.
Expect_BadLength(client)
This expects a BadLength error to be sent back from the X
server to client client.
Expect_BadIDChoice(client)
This expects a BadIDChoice error to be sent back from the X
server to client client.
Expect_Nothing(client)
This expects neither an error, event or reply to be sent
back from the X server to client client.
The Expect() function will check that the response
from the X server is of the correct type and has the correct
length. It will be byte swapped and unpacked as necessary
into an event or reply structure, to which a pointer will be
returned.
- 43 -
Programmers Guide for the X Test Suite
It is recommended that one of these functions be
called immediately after sending an X Protocol request to
the X server. This causes any pending response from the X
server to be flushed out, and checked. This makes it easier
to locate wrong responses from the X server. This is
effectively designing the test to run synchronously.
Once an error, event or reply has been returned, it can be
examined directly.
Since the structures allocated for requests, replies and
events are allocated dynamically, it is wise to free the
structure after use. this may be done using the functions
Free_Req(), Free_Reply() and
Free_Event().
When the outcome of sending the X Protocol request has been
assessed, you will want to either report an error or output
a trace message indicating that the expected response was
received. Refer to the earlier section entitled
"Outputting test information and result code".
You should end the test purpose if every part of the test
purpose has succeeded by calling Exit_OK(). This
should only be done once, because it is the means of passing
back to the library function testfunc() the fact
that the test purpose passed. If at an earlier part of the
test purpose an error occurs and it is desired to exit, call
Exit().
6.2.3.3 Creating test purposes to test X Protocol
extensions
The nature of the extension mechanism in X makes it
difficult to just add support in the switch statements
throughout the X Protocol library to support protocol
extensions.
The reason for this is that you do not know the value of the
event types and reply types until you have queried the X
server.
For this reason, you are recommended to review the scope of
the work that would be required in modifying the supplied X
Protocol library before attempting to test X Protocol
extensions. You can use the supplied X Protocol library as a
framework, and develop new versions of routines which handle
events and replies.
- 44 -
Programmers Guide for the X Test Suite
7. Source file libraries
This sections lists the contents of the principal libraries
of source files used by many tests in the X Test
Suite.
7.1 The X Test Suite library
A library of common subroutines for the X Test Suite has
source in $TET_ROOT/xtest/src/lib. This is built
automatically when building tests in the X Test Suite.
Should it be required to build it separately for any reason
run the command.
cd $TET_ROOT/xtest/src/lib
pmake install
The list of source files in this library, with a briefdescription of the contents of each file, is as follows: XTestExt.c If XTESTEXTENSION is defined, this file containsroutines to access the XTEST extension in orderto simulate device events and obtain informationon the cursor attributes of windows. If XTESTEXTENSION is not defined, dummy routinesare used instead. If XTESTEXTENSION is not defined, client-sidefunctions previously in file XTestLib.c (nowavailable in the XTEST extension library) arestill included. These are XTestDiscard() (todiscard current request in request buffer) andXTestSetGContextOfGC() andXTestSetVisualIDOfVisual() (to set values inopaque Xlib data structures). These functionsrequire access to data structures now in theinternal Xlib header file Xlibint.h. badcmap.c Create an invalid colourmap ID by creating areadonly colourmap of the default visual type. badfont.c Return a bad font ID by loading a font and thenunloading it. badgc.c Return a bad GC id on display disp by creating aGC and invalidating it using the XTEST extensionlibrary function XTestSetGContextOfGC. badpixm.c Return a bad pixmap id on display disp bycreating a pixmap and freeing it. badvis.c Make a visual bad by using the XTEST extensionlibrary function XTestSetVisualIDOfVisual. badwin.c Return a bad window id on display disp bycreating a window and destroying it.
- 45 -
Programmers Guide for the X Test Suite
bitcount.c
Handle bits in words.
block.c
Check whether process blocks when testing event handling
functions.
buildtree.c
Build a tree of windows specified by a list which determines
position, size and parentage of each window.
checkarea.c
Check pixels inside and/or outside an area of a drawable are
set to given values.
checkevent.c
Check two arbitrary events to see if they match, report an
error if they don’t.
checkfont.c
Check returned font characteristics, properties, text
extents and widths against those expected in the supplied
test fonts.
checkgc.c
Check GC components against expected values.
checkimg.c
Check pixels inside and/or outside an area of an image are
set to given values.
checkpixel.c
Check specified pixels of a drawable are set to given
values.
checktile.c
Check that an area of a drawable is filled with a specified
tile.
config.c
Initialise the config structure by getting all the execution
parameters.
crechild.c
Create a mapped child window for a parent window, and wait
for the child window to become viewable.
cursor.c
Routines for accessing cursor information. This includes
convenience functions for checking the cursor defined for a
given window. These routines call those in XTestExt.c to use
the XTEST extension to access the cursor
information.
delete.c
Set the test result code for the current test purpose to
UNRESOLVED .
devcntl.c
Routines for input device control. This includes convenience
functions for pressing keys and buttons and remembering
those pressed. These routines call those in XTestExt.c to
use the XTEST extension to simulate the
required device events.
dset.c
Set every pixel in a drawable to a specified
value.
- 46 -
Programmers Guide for the X Test Suite
dumpimage.c
Dump the contents of an image to a file.
environ.c
Contains a test suite specific version of putenv() (which
may not be available on POSIX.1 systems). This is required
to set up the environment before some calls to
tet_exec().
err.c
Test error handler (installed when calling the function
under test). Unexpected error handler (installed at all
other times). I/O error handler (installed at all times).
Obtain the error code and resource ID saved by the test
error handler.
events.c
Handle the serial fields of incoming requests.
ex_startup.c
Generic startup routines required before executing the first
test purpose and after executing the last test purpose. The
routines exec_startup() and exec_cleanup()
in this file should be called at the start and end of the
main() function of each program executed via the
TET function tet_exec().
exposechk.c
Check that either enough expose events were received to
restore the window, or that the window has been restored
from backing store.
extenavail.c
If XTESTEXTENSION is defined, the function
IsExtTestAvailable() returns True if the server extension
XTEST is available, otherwise it returns
False.
If XTESTEXTENSION is not defined, the
function IsExtTestAvailable() always returns False.
gcflush.c
Flush the GC cache.
gcinclude.c
Functions which are called from the code included to test
the correctness of use of GC components by the drawing
functions.
The only function included at present is
setfuncpixel(), which finds the first pixel set in
a drawable (this will vary depending on the drawing
function).
getevent.c
Check if there are events on the queue and if so return the
first one.
getsize.c
Get the size of a drawable. Just uses XGetGeometry but
avoids all the other information that you get with that.
gettime.c
Get the current server time. Use a property attached to the
root window of the display called
XT_TIMESTAMP and replace it with 42
(32-bits). The PropertyNotify event that is
- 47 -
Programmers Guide for the X Test Suite
generated supplies the time stamp returned.
iponlywin.c
Create an input only window.
issuppvis.c
The function issuppvis() takes a visual class as
argument and returns true if such a class is supported by
the server under test. This function uses the
XGetVisualInfo() function rather than the user-supplied
XT_VISUAL_CLASSES parameter.
The function visualsupported() takes a mask
indicating a set of visuals, and returns a mask indicating
the subset that is supported. If the mask is 0L then the
mask shows all supported visuals.
The function resetsupvis() takes a mask indicating
a set of visuals. Subsequent calls to nextsupvis()
will return the next supported visual specified in the mask
and increment a counter. The function nsupvis()
returns this counter.
linkstart.c
Define global variables used by the TET which are required
when linking test programs to produce a space-saving
executable.
When the space-saving executable is executed, the TET
initialisation code in the library function
linkstart.c determines which test set is required.
This is done by matching argv[0] with the
name elements in the array of linkinfo
structures. The corresponding test functions specified by
the testlist element of the linkinfo
structure are then executed.
lookupname.c
Convert symbolic values from X Window System header files to
appropriate names.
makecolmap.c
Make a colourmap for the screen associated with the default
root window.
makecur.c
Create a cursor that can be used within the test suite. The
cursor is created using XCreateFontCursor. The shape chosen
can be controlled through the configuration variable
XT_FONTCURSOR_GOOD .
makegc.c
Make a GC suitable for use with the given drawable.
makeimg.c
Creates a general purpose image that can be used within the
test suite. The image is cleared to W_BG.
makepixm.c
Creates a general purpose pixmap that can be used within the
test suite. The pixmap is cleared to W_BG.
- 48 -
Programmers Guide for the X Test Suite
makeregion.c
Creates a general purpose region that can be used within the
test suite.
makewin.c
Creates a general purpose windows that can be used within
the test suite.
makewin2.c
Creates windows corresponding to a particular area.
maxsize.c
Obtain the number of cells in a colourmap.
nextvclass.c
Functions to cycle through all the visual classes that are
supposed to supported by the display/screen that is being
tested. Note that these functions are only used in the tests
for XMatchVisualInfo and XGetVisualInfo.
The function initvclass() initialises the visual
class table. The visual classes that are supported are
supplied by the test suite user in the variable
XT_VISUAL_CLASSES , together with the depths
at which they are supported.
The function resetvclass() resets the visual class
table. Subsequent calls to nextvclass() obtain the
next visual class and depth. The function nvclass()
returns the size of the visual class table.
nextvinf.c
Functions to cycle through all the visuals that are
supported on the screen under test. These functions use the
XGetVisualInfo() function rather than the user-supplied
XT_VISUAL_CLASSES parameter. If the parameter
XT_DEBUG_VISUAL_IDS is set to a non-empty
string, only the visuals ID’s in the string are
used.
The function resetvinf() obtains a list of all
visuals supported for a particular screen. Subsequent calls
to nextvinf() obtain the next visual. The function
nvinf() returns the number of visuals.
nondpth1pix.c
Obtain a pixmap of depth other than 1 if such a pixmap is
supported.
notmember.c
Returns a list of numbers that are not members of given
list. (This is used to test assertions of the form
"When an argument is other than X or Y, then a BadValue
error occurs".
opendisp.c
Open a connection to the server under test.
openfonts.c
Open the xtest fonts, and place their ID’s in the
fonts array.
pattern.c
Draw a pattern consisting of vertical bands on the specified
drawable.
- 49 -
Programmers Guide for the X Test Suite
pfcount.c
Functions which may take arguments which are set to the pass
and fail counters in test set code created by mc.
Calls to the pfcount functions are inserted in
order to use the counters at least once, and so prevent
lint reporting unwanted errors.
pointer.c
Routines to move the pointer, and determine if the pointer
has been moved.
regid.c
Routines are provided to register resources created during a
test purpose. Wherever possible, library functions register
resources, and test purposes may do so directly if desired.
Registered resources are then destroyed at the end of the
test purpose.
report.c
Reporting functions, which output test information messages
to the TET journal file. These all use the TET reporting
function tet_infoline().
rpt.c
Reporting functions, which output test information messages
to the TET journal file, and additionally assign a test
result code. These all use the TET reporting function
tet_infoline().
savimage.c
The function savimage() returns a pointer to a
saved image on a drawable using XGetImage.
The function compsavimage() checks that the image
currently on the drawable matches a saved image.
The function diffsavimage() checks that the image
currently on the drawable differs from a saved image.
These functions are used where the precise pixels drawn
cannot be determined in advance, but the test result may
still be infered by image comparisons.
setline.c
Convenience functions to set line width, cap style, line
style and join style in a GC, using
XChangeGC().
settimeout.c
The function settimeout() sets a timeout which
causes the process to exit after a timeout. This should be
done only in a child process of a test purpose created by
tet_fork().
The function cleartimeout() clears a previously set
timeout.
stackorder.c
The function stackorder() uses
XQueryTree() to determine the position of a window
in the stacking order.
- 50 -
Programmers Guide for the X Test Suite
startcall.c
The function startcall() checks for any outstanding
unexpected X protocol errors, which might have been
generated, for example, during the setup part of the test. A
call to XSync() is made to achieve this.
The function startcall() installs a test error
handler in place of the unexpected X protocol error
handler.
The function endcall() checks for any X protocol
errors caused by the function under test. A call to
XSync() is made to achieve this.
The function endcall() installs the unexpected X
protocol error handler.
startup.c
Generic startup routines called by TET before executing the
first test purpose and after executing the last test
purpose.
tpstartup.c
Generic startup routines called by TET before executing each
test purpose and after executing each test purpose.
verimage.c
The function verifyimage() uses
XGetImage() to obtain the contents of the specified
drawable. This is then compared with the contents of a
"known good image file". If there is a
discrepancy, the image produced by the server is dumped to a
file using dumpimage() together with the known good
image. The image produced by the server and the known good
image may be examined as described in the section in the
"User Guide" entitled "Examining image
files".
If the execution configuration parameter
XT_DEBUG_NO_PIXCHECK is set to Yes,
the image checking is skipped in verifyimage().
If the execution configuration parameter
XT_SAVE_SERVER_IMAGE is set to Yes,
the image produced by the server is dumped to a file using
dumpimage() (regardless of whether it matches the
"known good image file").
For more background on pixmap verification see the earlier
section entitled "Creating test purposes which use
pixmap verification".
winh.c
Build a tree of windows to test event generation,
propogation and delivery.
xthost.c
Specifies operating system dependent data used by the access
control list functions. This includes arrays of
XHostAddress structures. These should be checked
and if necessary edited refering to the section in the
"User Guide" entitled "System dependent
source files".
xtestlib.h
This file contains definitions which are common to many of
the source files in the X Test Suite
- 51 -
Programmers Guide for the X Test Suite
library, and it is included in those source
files.
xtlibproto.h
This file contains declarations and (if required by an
ANSI Standard-C compiler) function prototypes
for all the functions in the source files in the X Test
Suite library.
7.2 The X Protocol library
A library of common subroutines for the X Protocol tests in
the X Test Suite has source in
$TET_ROOT/xtest/src/libproto. This is built
automatically when building tests in the X Test Suite.
Should it be required to build it separately for any reason
run the command.
cd $TET_ROOT/xtest/src/libproto
pmake install
The list of source files in this library, with a briefdescription of the contents of each file, is as follows: ClientMng.c Having established a client connection to the Xserver using the functions in ConnectMng.c,allocate a client data structure and fill in itsdisplay structure. ConnectMng.c Establish a client connection to the X server. DataMove.c Convert individual fields into format forsending to the X server. DfltVals.c Obtain reasonable default values for contents ofrequest structures. Expect.c Check for the expected response (error, event,reply, or nothing) from the X server. JustALink.c This file is a link to one of the filesXlibXtst.c, XlibOpaque.c, or XlibNoXtst.c. Thelink is created when the X Protocol library isbuilt, and the file used depends on theconfiguration parameter XP_OPEN_DIS. Log.c Log test results. MakeReq.c Construct a request structure using thefunctions in DfltVals.c, which has reasonabledefault values so that it may be immediatelysent to the X server using the functions inSendReq.c. RcvErr.c RcvEvt.c RcvRep.c Unpack the response from the server into astructure (RcvErr.c for errors, RcvEvt.c for
- 52 -
Programmers Guide for the X Test Suite
events, RcvRep.c for replies; these all use
DataMove.c to do the unpacking).
ResMng.c
Create a resource (e.g. atom, window) and store its resource
ID in the client data structure.
SendEvt.c
Pack an event structure into a request structure (only used
by SendEvent protocol request).
SendReq.c
Pack a request structure in correct format using the
functions in DataMove.c and send to the X server.
SendSup.c
Support routines for packing request structure.
ShowErr.c
ShowEvt.c
ShowRep.c
ShowReq.c
ShowSup.c
Display contents of structures in nice human-readable form
(ShowErr.c for errors, ShowEvt.c for events, ShowRep.c for
replies, and ShowReq.c for requests, all of which call
ShowSup.c support routines).
TestMng.c
Manage the setup and closedown of the tests. This file
includes definitions and initialisation of global variables
(including TET configuration variables) and assigning test
result codes.
TestSup.c
Support routines for handling mapping state and event masks
of windows.
Timer.c
Set up a timer that will execute a certain routine on
completion.
Utils.c
Utilities for isolating operating system dependencies.
Validate.c
Routines to check whether the server under test supports the
feature being tested (eg. writable colour cells).
ValListMng.c
Modify the value lists at the ends of request
structures.
XlibNoXtst.c
This file contains functions which emulate the post R5
enhanced connection setup scheme. A connection can be
established in client native or byte-swapped orientations,
and (when testing XOpenDisplay) both valid and invalid byte
orderings may be sent to the X server. The connection is
made using operating system specific procedures which were
developed in 4.2BSD environment, and may need modifications
to work on other systems.
- 53 -
Programmers Guide for the X Test Suite
XlibOpaque.c
This file contains portable functions to handle connection
setup where the Xlib implementation does not support the
post R5 enhanced connection setup scheme. The Xlib functions
XOpenDisplay and ConnectionNumber are called here to obtain
a connection using the client native byte orientation, and
subsequent X Protocol requests are made using this
connection.
XlibXtst.c
This file contains portable functions to handle connection
setup where the Xlib implementation supports the post R5
enhanced connection setup scheme. The enhancement involves
using additional parameters to the Xlib function
_XConnectDisplay() which allow a byte swapped connection to
be established. Details of operating system specific
connection setup procedures including networking are thus
not needed in the X Protocol library.
XstIO.c
Routines to handle protocol packet transmission and
reception including fatal I/O errors.
delete.c
Set the test result code for the current test purpose to
UNRESOLVED.
linkstart.c
Define global variables used by the TET which are required
when linking test programs to produce a space-saving
executable.
When the space-saving executable is executed, the TET
initialisation code in the library function
linkstart.c determines which test set is required.
This is done by matching argv[0] with the
name elements in the array of linkinfo
structures. The corresponding test functions specified by
the testlist element of the linkinfo
structure are then executed.
startup.c
Generic startup routines called by TET before executing the
first test purpose.
tpstartup.c
Generic startup routines called by TET before executing each
test purpose.
DataMove.h
This file contains the macros for byte swapping and word
alignment.
XstlibInt.h
This file contains definitions which are common to many of
the source files in the X Protocol library, and it is
included in those source files.
XstosInt.h
This file contains definitions related to operating system
functions which are common to many of the source files in
the X Protocol library, and it is included in those
source
- 54 -
Programmers Guide for the X Test Suite
files.
7.3 The X test fonts library
A library of common subroutines defining the characteristics
of the test fonts for the X Test Suite has source in
$TET_ROOT/xtest/fonts. This is built automatically
when building tests in the X Test Suite. Should it be
required to build it separately for any reason run the
command.
cd $TET_ROOT/xtest/fonts
pmake install
The source files xtfont0.c to xtfont6.c contain definitionsof XFontStruct structures named xtfont0 to xtfont6 whichdefine the characteristics of the test fonts used by many ofthe text drawing functions.
- 55 -
Programmers Guide for the X Test Suite
8. Appendix A - reason codes for extended
assertions
The reason code is a number between 1 and 6 (currently) and
is used if and only if the category is B or D. This number
corresponds to a reason from the following table which is
coded into mc.
The text of the reason will be printed with a result code
UNTESTED if there is no >>CODE.
1 - "There is no known portable test method for this
assertion",
2 - "The statement in the X11 specification is not
specific enough to write a test",
3 - "There is no known reliable test method for this
assertion",
4 - "Testing the assertion would require setup
procedures that involve an unreasonable amount of effort by
the user of the test suite.",
5 - "Testing the assertion would require an
unreasonable amount of time or resources on most
systems",
6 - "Creating a test would require an unreasonable
amount of test development time."
- 56 -
Programmers Guide for the X Test Suite
9. Appendix B - commands for fonts and symbols in
assertions
In the text of assertions there should be no in-line nroff
font changes. This is because the font names may need to be
changed on some systems.
As an alternative, a number of macros have been defined
which are understood by the utilities developed during stage
two of the project. The definition of these macros uses
appropriate fonts to correspond closely with those used by
the X Window System documentation.
1. Arguments to a function should be written:
.A window
2. Function names should be written:
.F XAllocColorCells
(When the special symbol xname is used it can be left as it
is, so the .F form only needs using when refering to some
other function. We have avoided cross references to other
functions where possible).
3. Structure members should be written:
.M override_redirect
4. Symbols should be written:
.S InputOutput
This is used for everything that is in the courier font in
the X Window System documentation and which is not a
function name or structure member. This includes the #define
constants in the headers and typedef’ed
names.
Eg.
BadColor
IsViewable
DirectColor
Visual
Display
MotionNotifyEvent
Punctuation separated by white space from the argument willbe in the original font, as in mm. .A InputOutput , .A InputOnly . —There is a .SM macro, as in mm. Any word that isuppercase only should use it to obtain a reduced pointsize. .SM DEBUG .SM MIT
- 57 -
Programmers Guide for the X Test Suite
10. Appendix C - Included error
assertions
The .ER keyword is described in the section entitled
"Included errors - .ER".
This appendix gives the names the files which are included
when this keyword is used with the supported arguments, and
shows the text of the assertions in those files.
All the files from which included tests are stored are
located in the directory
$TET_ROOT/xtest/lib/error.
The names of the files which are included, and the text of
the assertion contained in the file, are specified in the
following list:
Access grab
File included: EAcc1.mc
Assertion text:
When an attempt to grab a key/button combination already
grabbed by another client is made, then a BadAccess error
occurs.
Access colormap-free
|
File included: EAcc2.mc |
When an attempt to free a colormap entry not allocated by
the client is made, then a BadAccess error occurs.
|
Access colormap-store
|
File included: EAcc3.mc |
When an attempt to store into a read-only or an unallocated
colormap entry is made, then a BadAccess error occurs.
|
Access acl
|
File included: EAcc4.mc |
When an attempt is made to modify the access control list
from a client that is not authorised in a server-dependent
way to do so, then a BadAccess error occurs.
|
Access select
|
File included: EAcc5.mc |
When an attempt to select an event type is made, which at
most one client can select, and another client has already
selected it then a BadAccess error occurs.
|
Alloc
|
File included: EAll.mc |
When the server fails to allocate a required resource, then
a BadAlloc error occurs.
|
Atom [ARG1] [ARG2] ...
|
File included: EAto.mc |
When an atom argument does not name a valid Atom [, ARG1] [or
ARG2], then a BadAtom error occurs.
|
Color
|
File included: ECol.mc |
When a colourmap argument does not name a valid colourmap,
then a BadColor error occurs.
|
- 58 -
Programmers Guide for the X Test Suite
Cursor [ARG1] [ARG2] ...
File included: ECur.mc
Assertion text:
When a cursor argument does not name a valid Cursor [, ARG1] [or
ARG2], then a BadCursor error occurs.
|
Drawable [ARG1] [ARG2] ...
|
File included: EDra.mc |
When a drawable argument does not name a valid Drawable,
[ARG1] [or ARG2], then a BadDrawable error occurs.
|
Font bad-font
|
File included: EFon1.mc |
When a font argument does not name a valid font, then a
BadFont error occurs.
|
Font bad-fontable
|
File included: EFon2.mc |
When the font argument does not name a valid GContext or
font resource, then a BadFont error occurs.
|
GC
|
File included: EGC.mc |
When the GC argument does not name a defined GC, then a
BadGC error occurs.
|
Match inputonly
|
File included: EMat1.mc |
When a drawable argument is an InputOnly window then a
BadMatch error occurs.
|
Match gc-drawable-depth
|
File included: EMat2.mc |
When the graphics context and the drawable do not have the
same depth, then a BadMatch error occurs.
|
Match gc-drawable-screen
|
File included: EMat3.mc |
When the graphics context and the drawable were not created
for the same root, then a BadMatch error occurs.
|
Match wininputonly
|
File included: EMat4.mc |
When the window argument is an InputOnly window then a
BadMatch error occurs.
|
Name font
|
File included: ENam1.mc |
When the specified font does not exist, then a BadName error
occurs.
|
Name colour
|
File included: ENam2.mc |
When the specified colour does not exist, then a BadName
error occurs.
|
Pixmap [ARG1] [ARG2] ...
|
File included: EPix.mc |
- 59 -
Programmers Guide for the X Test Suite
Assertion text:
When a pixmap argument does not name a valid Pixmap [, ARG1] [or
ARG2], then a BadPixmap error occurs.
|
Value ARG1 VAL1 [VAL2] ...
|
File included: EVal.mc † |
When the value of ARG1 is other than VAL1 [or VAL2], then a
BadValue error occurs.
|
|
† - the assertion text is not in the included file, but is inserted directly by mc. |
Value ARG1 mask VAL1 [VAL2] ...
|
File included: EVal.mc † |
When the value of ARG1 is not a bitwise combination of VAL1
[or VAL2], then a BadValue error occurs.
|
|
† - the assertion text is not in the included file, but is inserted directly by mc. |
Window [ARG1] [ARG2] ...
|
File included: EWin.mc |
When a window argument does not name a valid Window [, ARG1] [or
ARG2], then a BadWindow error occurs.
|
- 60 -
Programmers Guide for the X Test Suite
11. Appendix D - mc
utility
Usage
mc [-a a_list] [-o <output-file>] [-l] [-m] [-s] [-p] [<input-file>]
The mc utility outputs a C source file containing testsspecified in the input file <input-file>, which must be adot-m file which has the format specified in the sectionentitled "Source file syntax". If no <input-file> is specified, the input is taken fromstandard input. Multiple input files can be processed bythe utility, but the overall syntax must still conform tothat defined in the section entitled "Source file syntax".A consequence of this is that you cannot specify anothertitle section for a different function and expect to outputtests for more than one function simultaneously. Limiteddiagnostics are given if the file does not have the requiredsyntax. By default, the C source file is written to thestandard output stream. More details of the formats of the C source files producedby mc are given in the section entitled "Source fileformats". Options -a a_list This permits the specification of a list of assertionsof the form n1-m1,n2-m2,... to be output. Test codewill only be output corresponding to the tests in thespecified ranges. -o output-file This sends the output to the file <output-file> insteadof the standard output stream. -l This option outputs a C source file containing testswith modified startup code which allows the source codeto be compiled and linked into a space-savingexecutable file. The format of these files is describedin the section entitled "C files for linked executable- link.c". -m This option outputs a C source file containing testsfor the macro version of the function specified in thetitle section of the dot-m file. The format of thesefiles is described in the section entitled "C files forstandalone executable in macro tests - MTest.c". The macro name is set to the <macroname> argument ofthe >>SET macro option - if there is no >>SET macrooption in the file, or no argument specified, thedefault is the function argument in the >>TITLEkeyword, with the leading letter ‘X’ removed. -s This option outputs a test strategy from the dot-m fileas a C source code comment block between the assertionand the code. The test strategy is derived from the
- 61 -
Programmers Guide for the X Test Suite
corresponding strategy section in the dot-m
file.
-p
This causes additional output including indicators of line
number in the original dot-m file (where possible). This
means that any diagnostics produced by cc(1) or
lint(1) will refer to the line numbers in the
original dot-m file rather than the C source
file.
- 62 -
Programmers Guide for the X Test Suite
12. Appendix E - mmkf
utility
Usage
mmkf [-o <output-file>] [-s sections] [<input_file>]
The mmkf utility outputs a Makefile corresponding to thespecified input file <input-file>, which must be a dot-mfile which has the format specified in the section entitled"Source file syntax". The Makefile can build all the Csource files that can be output by mc from the input file<input-file>. If no <input-file> is specified, the input is taken fromstandard input. Multiple input files can be processed bythe utility, but the overall syntax must still conform tothat defined in the section entitled "Source file syntax".A consequence of this is that you cannot specify anothertitle section for a different function and expect to outputMakefiles for more than one function simultaneously.Limited diagnostics are given if the file does not have therequired syntax. By default, the Makefile is written to thestandard output stream. More details of the formats of the Makefiles produced bymmkf are given in the sub-section entitled "Makefile" in thesection entitled "Source file formats". Options -o output-file This sends the output to the file <output-file> insteadof the standard output stream. -s sections This option enables output of certain optional sectionsof the Makefile. By default, output of all thesesections is enabled. There is no reason why you shouldneed to use this option with the current version of theX Test Suite. The sections argument is a character string which maycontain the key letters l, L, m and p. If thesecharacters are included, the specified sections of theMakefile are then output.
Key Optional
letter section
l Targets for linked executable
- 63 -
Programmers Guide for the X Test Suite
L Targets for libraries
m Targets for linting and cleaning
p Targets for building known good image
files
- 64 -
Programmers Guide for the X Test Suite
13. Appendix F - ma
utility
Usage
ma [-a a_list] [-o <output-file>] [-h] [-s] [-p] [-m] [<input-file>]
The ma utility outputs a file containing a list ofassertions in nroff(1) format (requiring no macros otherthan those supplied in file maheader.mc). The assertionsare specified in the input file <input-file>, which must bea dot-m file which has the format specified in the sectionentitled "Source file syntax". If no <input-file> is specified, the input is taken fromstandard input. Multiple input files can be processed bythe utility, but the overall syntax must still conform tothat defined in the section entitled "Source file syntax".A consequence of this is that you cannot specify anothertitle section for a different function and expect to outputassertions for more than one function simultaneously.Limited diagnostics are given if the file does not have therequired syntax. By default, the assertion list is writtento the standard output stream. More details of the format of the assertion list produced byma are given in the sub-section entitled "Formattingassertions" in the section entitled "Source file formats". Options -a a_list This permits the specification of a list of assertionsof the form n1-m1,n2-m2,... to be output. Assertionswill only be output corresponding to the tests in thespecified ranges. -o output-file This sends the output to the file <output-file> insteadof the standard output stream. -h The macros required for formatting the assertions areincluded at the start of the output stream. These arecopied from the file maheader.mc. By default, the macros are not copied to the outputstream. -s If this option is specified, and the -h option isspecified, the line
.so head.t
will be output at the start of the output stream. This option is not intended for general use - it wasused when distributing assertions in compact form forexternal review. -p The macros .NS and .NE will be output before and after
- 65 -
Programmers Guide for the X Test Suite
each line in the dot-m file which is a comment
(commencing with >>#). By default, dot-m file comments
are not output. The macros .NS and .NE are
defined in maheader.mc; they cause the dot-m file
comment lines to be printed in italic font by
nroff(1).
This option is not intended for general use - it was used
when reviewing assertions before delivery.
-m
This option outputs assertions for the macro version of the
function specified in the title section of the dot-m
file.
The macro name is set to the <macroname>
argument of the >>SET macro option - if there is no
>>SET macro option in the file, or no argument
specified, the default is the function argument in
the >>TITLE keyword, with the leading letter
‘X’ removed.
CONTENTS
1.
Introduction
...................................... 1
2.
Purpose of this guide
............................. 1
3.
Contents of this guide
............................ 1
|
3.1 |
4.
Source file syntax
................................ 3
|
4.1 |
|
4.2 |
|
4.3 |
|
4.4 |
|
4.5 |
|
4.6 |
|
4.7 |
|
4.8 |
|
4.9 |
|
4.10 |
|
4.11 |
5.
Source file formats
.............................. 13
|
5.1 |
|
5.2 |
|
5.3 |
|
5.4 |
|
5.5 |
|
5.6 |
6.
Source file structure
............................ 32
|
6.1 |
|
6.2 |
7.
Source file libraries
............................ 45
|
7.1 |
|
7.2 |
|
7.3 |
8.
Appendix A - reason codes for extended assertions
....................................... 56
9.
Appendix B - commands for fonts and symbols in
assertions
....................................... 57
10.
Appendix C - Included error assertions
........... 58
11.
Appendix D - mc utility
.......................... 61
12.
Appendix E - mmkf utility
........................ 63
13.
Appendix F - ma utility
.......................... 65
i