An example of a software test plan is the DoD-STD-2167A, A test plan attached at the end of this lesson. This is called a data item description (DID) and has an identification number DI-MCCR-80014A.
Test Plan Preparation
The image below, Test Plan Preparation shows the activities necessary
to build the Software Test Plan (STP) document.
There are specific types of documentation necessary as input to the test plan preparation. The documents may vary depending on the software project. The typical documents are the Software Test Plan, Software Requirements Specification, the Detailed Design Specification and the Master Test Plan. Other projects may include the Functional Specifications, Pseudo-design Language (PDL) and, a Graphical User Interface screens template document.
The Software Test Plan contains the logical grouping of the test requirements.
The Test Engineer should consider the following when grouping the requirements:
- Qualification methods (such as Analysis, Demonstration, Inspection, etc.)
- Test methods (such as boundary, equivalence class partitioning, etc.)
- Types of testing such as static or dynamic
- Development method:
- Object Oriented vs. Functional
- Single build vs. incremental
The Software Test Plan will be peer reviewed with the Test Engineering being the author. The two military standards DOD-STD-2167A or DOD-STD-498 are another source for a description of test definitions. The Rational Unified process also has an example set of template test definitions.
Test Case Preparation
The image below, Test Case Preparation, shows a partial list of items to build the test cases.
Test Case Preparation
|
The contents and formats for the Test Cases can be found in the Software Test Report of MIL-STD-2167A, DID DI-MCCR-80017A which is enclosed at the end of this lesson. The typical contents are: the test set-up; the resources needed to execute the test cases; the step-by-step control actions and expected results and, the pass or fail of expected results.
Design Issues & Tailoring. Each project will typically have its own design issues and tailor the test case to its own format. This list should be considered and is not intended to be a complete list:
- Standards (IEEE, DoD, etc.)
- Additional information
- Test log
- Witness Signatures
- Date & time executed
- Platform
Prior to any formal test execution a series of dry runs are performed without Quality Assurance personnel observing officially. The goal of the dry run is to verify that test cases are procedurally correct and to detect all defects so that a minimal number of procedural problems and defects are found during Formal Test execution. Three levels of dry run should be considered. The author or responsible test engineer must dry run the procedure/scripts without any witness, no matter what. A dry run by another tester, using the author’s procedures, if the Formal Test is witnessed by client, by the test engineer with Quality Assurance personnel or Security as the witness. Security (as applicable) should take part or if Formal Test is witnessed by the client. QA is an independent function that typically monitors the dry run testing function. QA will stamp the Formal Test procedures and initial each step of the test procedures as they are successfully demonstrated. The image below, Dry Run Activities depicts some of the activities involved in running the dry run of the formal tests.
The dry run activity is a process to execute each test case. The test conductor should log each problem/defect in a test log procedures notebook. The test conductor should mark red-line to the cases and incorporate the red-lines into the formal test procedures. The test conductor should retest resolved problems/defects and retest the new red-lined cases. The retest should be conducted by the author of the test case.
The image below, Test Execution Activities, depicts some of the test execution activities involved in running the formal tests as observed by the witness personnel.
The formal test execution activities are not extremely different from the dry run except for obtaining QA witness signatures and stamp of approval. The signatures and stamp of approval are dependent on the type of the software product and the contract statement. The test schedule is very important to be established with QA for completing test execution. A backup plan should also be established based on personnel schedules, holidays or additional hardware. The Test Manager needs a mitigation plan for schedule slip because something seems to always not go as planned.