Guidelines: Test Script

Test Script

Test Scripts are the computer readable instructions that automate the execution of one or more Test Cases (or portions of ). Test Scripts may be created (recorded ) or automatically generated using test automation tools, programmed using a programming language, or a combination of recording, generating, and programming.

Topics

Structure of Test Scripts

To increase the maintainability and reusability of your Test Scripts, they should have been structured before they are implemented. You will probably find that there are actions that will appear in several Test Scripts. A goal should be to identify these actions so that you can reuse their implementation.

For example, you may have Test Scripts that are combinations of different actions you can perform to a record. These Test Scripts may be combinations of the addition, modification, and the deletion of a record:

If you identify and implement these actions as separate Test Scripts and reuse them in other Test Scripts you will achieve a higher level of reuse.

Another goal would be to structure your Test Scripts in such a way that a change in the target software causes a localized and controllable change in your Test Scripts. This will make your Test Scripts more resilient to changes in the target software. For example, say the login portion of the software has changed. For all test cases that traverses the login portion, only the Test Script pertaining to login will have to change.

Recording Technique

To achieve higher maintainability of your test scripts, you should record them in a way that is least vulnerable to changes in the target-of-test. For example, for a test script that fills in dialog box fields, there are choices for how to proceed from one field to the next:

Of these choices, some are more vulnerable to design changes than others. If a new field is inserted on the screen the TAB key approach will not be reliable. If accelerator keys are reassigned, they will not provide a good recording. If the method that the mouse uses to identify a field is subject to change, that may not be a reliable method either. However, some test automation tools have test script recorders that can be instructed to identify the field by a more reliable method, such as its Object Name assigned by the development tool (PowerBuilder, SQLWindows, or Visual Basic). In this way, a recorded test script is not effected by minor changes to the user interface (e.g., layout changes, field label changes, formatting changes, etc.)

Data-Driven Testing

Many Test Scripts involve entering several sets of field data in a given data entry screen to check field validation functions, error handling, and so on. The procedural steps are the same; only the data is different. Rather than recording a Test Script for every set of input data, a single recording should be made and then modified to handle multiple data sets. For example, all the data sets that produce the same error because of invalid data can share the same recorded Test Script. The Test Script is modified to address the data as variable information, to read the data sets from a file or other external source, and to loop through all of the relevant data sets.

If Test Scripts or test code have been developed to loop through sets of input and output data the data sets must be established. The usual format to use for these data sets is records of comma-separated fields in a text file. This format is easy to read from Test Scripts and test code, and is easy to create and maintain.

Most database and spreadsheet packages can produce comma-separated textual output. Using these packages to organize or capture data sets has two important benefits. First, they provide a more structured environment for entering and editing the data than simply using a text editor or word processor. Second, most have the ability to query existing databases and capture the returned data, allowing an easy way to extract data sets from existing sources.

Error Handling

The recorded Test Script is sequential in its execution. There are no branch points. Robust error handling in the Test Scripts requires additional logic to respond to error conditions. Decision logic that can be employed when errors occur includes:

Each error-handling technique requires program logic added to the Test Script. As much as possible, this logic should be confined to the high-level Test Scripts that control the sequencing of lower-level Test Scripts. This allows the lower-level Test Scripts to be created completely from recording.

Test Script Synchronization and Scheduling

When doing stress testing, it is often desirable to synchronize Test Scripts so that they start at predefined times. Test Scripts can be modified to start at a particular time by comparing the desired start time with the system time. In networked systems each test station will share, via the network, the same clock. In the following example (from a script written in Visual Basic) statements have been inserted at the start of a script to suspend the execution of the script until the required time is reached.

InputFile$ = "\TIME.DAT"
Open InputFile$ For Input As 1
Input #1, StartTime$
Close #1
Do While TimeValue(StartTime$) > Time
DoEvents
Loop
[Start script]

In this example, the required start time is stored in a file. This allows the start time to be changed without changing the Test Script. The time is read and stored in a variable called StartTime$. The Do While loop continues until the starting time is reached. The DoEvents statement is very significant. It allows background tasks to execute while the Test Script is suspended and waiting to start. Without the DoEvents statement, the system would be unresponsive until the start time had been reached.

Testing and Debugging Test Scripts

When the newly recorded Test Scripts are executed on the same software on which they were recorded, there should be no errors. The environment and the software are identical to when it was recorded. There may be instances where the Test Script does not run successfully. Testing the Test Scripts uncovers these cases, and allows the scripts to be corrected before being used in a real test. Two typical kinds of problems are discussed here:

Timing differences in recording and playback can lead to problems. Recording a Test Script is inherently a slower process than executing it. Sometimes this time difference results in the Test Script running ahead of the software. In these cases, Wait States can be inserted to throttle the Test Script to the speed of the software.