Roles and Activities > Developer Role Set > Test Designer > Develop Test Guidelines

Activity:  Develop Test Guidelines

Identify and obtain existing guidelines you can reuse

In Inception or early Elaboration, you should take some time out to locate existing guidelines. It's useful to think of Guidelines as falling into one of three general categories: project control and enactment (either strategic or tactical), standards adherence and project-specific (or organization-specific) practices. When looking for existing guidelines, they will usually fit one of these categories.

1. Project control and enactment

These are guidelines that document the strategic or tactical decisions taken on a specific project. Having a look through the planning documents of current or historical projects is a good place to find these guidelines. If you also have access to the assessment documents for the same project, you might be able to form an opinion of how helpful or successful the guidelines were.

2. Standards Adherence

Standards such as IEEE Std829-1998 [IEEE] provide templates and examples for adhering to a documentation set with specific format. In the case of Std829, it relates specifically to the test documentation set. RUP provides various artifact templates that are suitable for use when applying many of the IEEE document standards. The SEI CMM and ISO9001 provide a standard by which the maturity of or adherence to the enactment of a software process can assessed.

Other categories of standard include certification for specific domains or environments. RTCA's DO-178B standard describes Software Considerations in Airborne Systems and Equipment Certification (See the RTCA web-site for more information).

Some organizations also have their own internal standards that need to be adhered to. Typical categories include programming conventions and style guides, user-interface standards and style guides, and compliance with organization-wide architectural conventions, constraints and standards. Examples include use of centralized audit, security or transaction control services, constraints placed on the language features and techniques used, or limits on the architecture itself (fat-clients are forbidden, basic HTTP protocol is considered a security risk etc.

A search using most standard Internet search engine will point you to a large number of resources.

3. Project-specific (or organization-specific) practices

This type of process tailoring or adjustment is often relatively informal in nature. The guidelines themselves typically take the form of heuristic guidance or "cookbooks", allowing compliance by the practitioner on a subjective "not-enforced" basis. For this guidance to be "consumable", it's important to strive for a an optimum balance between prescriptive guidance and descriptive explanation. While arguably the most valuable of the categories of guidelines, the situational context and specialized nature of best-practices often make them difficult to find in a documented, reusable form.

RUP provides many good software engineering practices that can be accessed through the RUP web-site product. Two specific parts of RUP rich in good practice are the artifact Guideline and Checkpoint pages (e.g.Checkpoints: Test Plan).

Some web communities also provide resources related to good practices, such as the software patterns community. Here is a partial listing of web community resources with a specific focus on testing:

• TestingCraft: A site hosted by Brian Marick, where Software Testers Share Techniques.
• Cem Kaner: Articles on Dr. Cem Kaner's observations of testing techniques, etc.
• Satisfice: Various articles by James Bach and others on testing issues.
• Patterns Of Software Test (PoST): A site hosted by Brian Marick, where the application of Patterns are explored and discussed in relation to the Testing of Software.

Perform initial tailoring

Consider what initial tailoring is required to suit the scope of the project and the corresponding test effort. Some tailoring will be required based on team size, other tailoring will be based on appropriate process and organization culture.

Identify the appropriate guidelines you plan to use

Guidelines tend to be created as catchall "encyclopedia" style works, often a as a single artifact. The other end of the spectrum are large numbers of small, focused notes on specific concerns. Both these formats can make referencing and using guidelines time-consuming and distracting.

Consider creating a reference artifact for the project that includes pointers to just the guidelines you feel are applicable in the current context. You'll need to allow time to update this moving forward. Avoid duplicating too much in the reference artifact: try to limit the entries to an identifying name and a locator.

Decide how you will tailor the guidelines, if at all

Give thought to ways in which the guidelines might be both extended and reduced to best suit the needs of the team.

Once you've tabled all of the ideas for tailoring, gain consensus opinion on what tailoring will be performed.

Capture tailoring decisions

Once you've decided what guidelines are appropriate in the current context, document these decisions. You'll probably want to use two formats: one to record the decisions you've made, and another to record the Test Guidelines themselves, their methods and descriptions.

The development case serves as one format for recording the decisions made regarding what guidelines to follow, as do the software development plan, the test plan(s) and other project management related documents.

For additional information, see the section "Tailoring" in Test Guidelines.

Verify and revise guidelines

Review and verify that the guidelines are written clearly and consistently, and revise the guidelines accordingly so that they will provide the appropriate guidance to the teams that use them.

Capture project-specific practices

Encourage team members to identify and note project-specific practices. The software patterns community has given birth to the WikiWikiWeb, an excellent mechanism for sharing practitioner experience. The oldest public wiki (the Portland Pattern Repository) was created in 1995, and remains active as of July 2001. You can visit the PPR at Wiki:WelcomeVisitors. We recommend you consider creating your own in-house WikiWikiWeb to capture and develop Project-specific (or organization-specific) practices. For information on getting started with your own Wiki, see WikiWikiWeb.

Even if you cannot or do not use the WikiWeb format, there are some things to remember. First, it is important to provide a mechanism that encourages good practices to be identified and simply recorded. This mechanism needs to allow the practice to be identified, it's basic method and it's key benefits recorded easily. This basic information can then be expanded and presented by the staff in the process engineering role. Avoid create a bureaucratic process and documentation overhead that discourages team members from contributing.

Learn from other projects and organizations

Most good practices generally become good because they've been used a number of times before and have had time be fine-tuned and improved.

You don't always have to reinvent the wheel: take advantage of both the success' and failures of other projects and organizations by making use of their guidelines.

Improve the guidelines

As the guidelines are used, new insights will be gained on what is most important about them, and what aspects are less relevant. Make time in each build cycle or iteration to revise and further develop the guidelines.

To encourage and facilitate this process, provide an easy-to-use forum that allows team members to quickly record potential improvements as they are identified.

Evaluate and verify your results

Now that you have completed the work, it is beneficial to verify that the work was of sufficient value, and that you did not simply consume vast quantities of paper. You should evaluate whether your work is of appropriate quality, and that it is complete enough to be useful to those team members who will make subsequent use of it as input to their work. Where possible, use the checklists provided in RUP to verify that quality and completeness are "good enough".

Have the people performing the downstream activities that rely on your work as input take part in reviewing your interim work. Do this while you still have time available to take action to address their concerns. You should also evaluate your work against the key input artifacts to make sure you have represented them accurately and sufficiently. It may be useful to have the author of the input artifact review your work on this basis.

Try to remember that that RUP is an iterative process and that in many cases artifacts evolve over time. As such, it is not usually necessary—and is often counterproductive—to fully-form an artifact that will only be partially used or will not be used at all in immediately subsequent work. This is because there is a high probability that the situation surrounding the artifact will change—and the assumptions made when the artifact was created proven incorrect—before the artifact is used, resulting in wasted effort and costly rework. Also avoid the trap of spending too many cycles on presentation to the detriment of content value. In project environments where presentation has importance and economic value as a project deliverable, you might want to consider using an administrative resource to perform presentation tasks.