Tool Mentor: Executing Test
Suites Using Rational Robot
Purpose
This tool mentor describes how to use Rational Robot to execute Test Suites
(playing back one or more Test Scripts), and to analyze the results of the test
execution.
Related Rational Unified Process information:
Overview
This tool mentor is applicable when running Microsoft Windows 95/98/2000/NT 4.0.
To execute the test scripts recorded or programmed using Rational Robot, play back the
test script.
- Play the script back
- View the results in the TestManager log
- Analyze verification point results with the Comparators
When you play back a script, Rational Robot repeats your recorded actions and automates
the execution of the test. With automation, you can test each new build of your
application faster and more thoroughly than by testing it manually. This decreases testing
time and increases both coverage and overall consistency.
Test scripts are played back at several times during the test life cycle:
- Test Implementation Play back the scripts to verify that they work as
intended, using the same build of the application-under-test that you used to record. This
verifies the baseline of expected behavior for the application-under-test.
- Test Execution and Regression Play back the scripts to compare the latest
build of the application-under-test to the baseline established during test
implementation. Ongoing play back of all test scripts during the development cycle reveals
any differences that may have been introduced into the application since the last build.
These differences can be evaluated to determine if they are actual defects or deliberate
changes.
Before playback, Rational Robot automatically compiles a Test Script if it
has changed since it last ran.
For information on preparing the test environment for Test Script playback,
see Tool Mentor: Setting up the Test Environment in Rational
Robot.
To play back a Test Script:
- Click the Playback Script button on the toolbar. The Playback dialog
box will appear.
- Type a Test Script name or select it from the list.
To change the Playback options, click GUI Options.
When finished, click OK. For information on playback options, see Tool Mentor: Setting up the Test Environment in Rational Robot.
- Click OK to continue.
- If the Specify Log Information dialog box appears, do the following:
- Select a Build from the list. To create a new Build, click the Build
button on the right.
- Select a Test Log folder from the list. To create a new Test Log folder,
click the Log Folder button on the right.
- Accept the default Test Log name, which is the same as the Test Script
name, or type a new name.
- Click OK.
- If a prompt appears asking if you want to overwrite the log, click Yes.
Note: To stop playback of a Test Script, press the F11 key. Rational
Robot recognizes the F11 key only when playing back Object-Oriented commands.
The F11 key does not stop Test Script playback during low-level actions.
After playback completes, use the TestManager Test Log to view the playback
results, including verification point failures, procedural failures, aborts,
and any additional playback information.
To control the Test Log information and display of the Test Log, you can set
options in the Log tab in the GUI Playback Options dialog box:
- To update your project with the playback results, select Output playback results to
log.
- To display the log automatically after playback, select View log after playback.
If this is not selected, you can display the Test Log after playback by clicking
Tools > Rational Test > Rational TestManager.
If the playback of the Test Scripts resulted in any errors, the specific cause
of the error must be identified.
Test Scripts which failed to execute sucessfully are identified in the Test
Log by the word Fail (shown in red) in the Results
column, or Warning (shown in yellow). There are several
classes of execution failure to be addressed: fatal errors, script errors, and
verification point failures. Fatal errors often indicate that there is a problems
with the Test Environment Configuration rather than the Test Script itself.
Investigate the Test Environment Configuration and isolate the failure and resolve
the corresponding fault. Resolving Test Script errors may require some Test
Script maintenance. Verification point failures (during test implementation)
typically require changes to the verification point parameters or indicate a
difference between the desired state of the Target Test Items when the script
was recorded versus when it was played back. See the next section for analyzing
verification points.
Additional information regarding the error condition may be viewed in the Log
Event Properties dialog box. Open this box by selecting a Test Log
event and clicking View > Properties.
For each Test Script in which a failure (or warning) occurred, review the information
regarding the failure and if necessary, edit the Test Script (see Section
3. Edit the test script if necessary above). The Test Script (and Rational
Robot) may be opened from the Test Log by clicking View > Script.
After editing the Test Script, the environment should be re-set to its appropriate
initial state (see Tool Mentor: Setting Up the Test Environment
in Rational Robot and the Test Suites re-executed (Play
the script back).
Use the Comparators to analyze verification point results. To open a Comparator
from the TestManager Test Log:
- In the Log Event column, select a verification point and click View > Verification
Point. The appropriate Comparator will appear. The Comparators may also be opened from
Rational Robot by double-clicking a verification point in the Asset pane. However, when
you open a Comparator this way, you can view only the baseline file.
- Differences between the baseline and the actual results will be highlighted.
- Review the differences and determine the appropriate course of action, including:
(Note: the available actions will differ based upon the type of verification
point.)
- Copy property to baseline copy the current individual actual property and
over-write the baseline property.
- Copy all properties to baseline copy all the actual property values and
over-write all the baseline properties.
- Edit individual property (menu, value, and so on) by double-clicking the property (menu,
value, and so forth) and entering a different value.
- Editing the properties list (to identify which properties should be captured by the
verification point).
- Create or modify masks.
- When finished making the appropriate changes in the Comparator, click File > Save
Baseline to save the changes, and click the Exit button to close the
Comparator.
After editing the Test Script or verification points, the environment should
be reset to its appropriate initial state (see Tool Mentor:
Setting Up the Test Environment in Rational Robot) and the tests re-executed
(Play the script back).
Copyright
© 1987 - 2001 Rational Software Corporation
|