Pete Sawyer
Software requirements engineering is of great economic importance to the software industry. The cost of fixing errors in the requirements tends to increase exponentially the longer they remain undetected. Projects waste enormous resources rewriting code that enshrines mistaken assumptions about customers' needs and environmental constraints. There is therefore considerable risk involved in software requirements engineering yet the maturity of most software development organisations' requirements engineering processes lags well behind that of their downstream life-cycle processes. This is compounded by the fact that software requirement engineering is tightly time- and resource-bounded and time spent analysing requirements is often wrongly perceived to be unproductive time.
The issue of who should perform software requirements engineering is controversial. Requirements engineering requires a set of skills that intersect with those of software engineering but which also includes skills outside of this set. For this reason, we refer in this document to the 'requirements engineer' rather than the 'software engineer'. This is not to suggest that software engineers cannot 'do' requirements engineering, but rather to emphasise that the role of the requirements engineer is a distinct one that mediates between the domain of software development and the domain of the software's customer(s). Requirements engineering may not be the preserve of software engineers but it should be performed by people with software engineering knowledge.
It is useful to consider the meaning of the term 'software requirement'. At its most basic, a software requirement can be understood as a property that the software must exhibit in order for it to adequately perform its function. This function may be to automate some part of a task of the people who will use the software, to support the business processes of the organisation that has commissioned the software, controlling a device in which the software is to be embedded, and many more. The functioning of the users, or the business processes or the device will typically be complex and, by extension, the requirements on the software will be a complex combination of requirements from different people at different levels of an organisation and from the environment in which the software must execute.
Clearly, requirements will vary in intent and in the kinds of properties they represent. A crude distinction is often drawn between:
- Capabilities that the software must provide, such as saving a file or modulating a signal. Capabilities are sometimes known as functional requirements.
- Constraints that the software must operate under, such as having a probability of generating a fatal error during any hour of operation of less than 1 * 10-8. Constraints are sometimes known as non-functional or quality requirements.
In practice, this distinction is hard to apply rigidly and it is difficult to formulate a consistent definition of the distinction between them. Constraints are particularly hard to define and tend to vary from vaguely expressed goals to very specific bounds on the software's performance. Examples of these might be: that the software must increase the call centre's throughput by 20%; and that the task will return a result within 10ms. In the former case, the requirement is at a very high level and will need to be elaborated into a number of specific capabilities which, if all satisfied by the software, should achieve the required improvement in throughput. In the latter case, the requirement will constrain the manner in which the software engineers implement the functionality needed to compute the result.
This also illustrates the range of levels at which requirements are formulated. The standard, though rather unsatisfactory, way of distinguishing levels of requirements is as 'user' or 'system' requirements. User requirements are the direct requirements of the people who use or otherwise have a stake in (the 'stakeholders') the software. They consider the software to be a 'black box' and are only concerned with what is externally exhibited by the software. Their collection, organisation and analysis is crucial to gaining a clear understanding of the problem for which the software is to provide a solution and its likely cost. This understanding has to be validated and trade-offs negotiated before further resources are committed to the project. The user requirements therefore need to be expressed in terms that allow the stakeholders to do this validation. This usually means the requirements are expressed using a structured form of natural language to describe the software's effect on the stakeholders' business domain. Exceptionally, this may be supplemented by models using specialised software engineering notations, but used carefully so that they can be understood with minimal explanation.
Software engineers who are experts in the stakeholders' domain are very rare so they cannot be expected to take a list of user requirements, interpret their meaning and translate them into a configuration of software components that satisfy the user requirements. In most cases, therefore, user requirements are elaborated by the requirements engineer into a number of more detailed requirements that more precisely describe what the software must do. This usually entails deploying engineering skills to construct models of the system in order to understanding the logical partitioning of the system, its context in the operational environment and the data and control communications between the logical entities. A side-effect of this is that new requirements (emergent properties) will emerge as a conceptual architecture of the software starts to take shape. It is also likely to reveal problems with the user requirements (such as missing information) which have to be resolved in partnership with the stakeholders.
The result of this process is a set of detailed and more rigorously defined requirements. These are the software requirements (or 'system requirements'). Because the intended readership is technical, the software requirements can make use of modeling notations to supplement the textual descriptions of the software requirements. The software requirements enable a system architect to propose a solution architecture. They form the baseline for the real development work so they should enable detailed planning and costs to be derived.
The above description implies a deterministic process where user requirements are elicited from the stakeholders, elaborated into software requirements and passed over to the development team. This is an idealised view. In practice, many things conspire to make software requirements engineering one of the riskiest and most poorly understood parts of the software life-cycle. The requirements engineer may interpret the stakeholders' requirements wrongly, the stakeholders may have difficulty articulating their requirements, or technology, market conditions or the customer's business processes may change midway through development.
All of these things militate against the ideal of having a requirements baseline frozen and in place before development begins. Requirements will change and this change must be managed by continuing to 'do' requirements engineering throughout the life-cycle of the project. In a typical project the activities of the requirements engineer evolve over time from elicitation and modeling to impact analysis and trade-off negotiation. To do this effectively requires that the requirements are carefully documented and managed so that the impact of proposed changes can be traced back to the affected user requirements and forwards to the affected implementation components.
What we have described above refers to requirements on the software (the 'product'). Requirements can also be imposed on how the software is developed (the 'process'). Process requirements are also important because they can impact on development schedules, development costs and the skills required of the developers. Thus, it is important to (for example) understand the implications of working for a customer who requires that the software contractor has achieved accreditation to CMM level 3. An important process requirement that every project has is a cost constraint.
There is no universally accepted model of the requirements engineering process that will suit all organisations or all projects. The process will vary for many reasons but among the factors that have the most influence are:
- The nature of the project. A market-driven project that is developing a software product for general sale imposes different demands on the requirements engineering process than does a customer-driven project that is developing bespoke software for a single customer. For example, a strategy of incremental releases is often used in market-driven projects while this is often unacceptable for bespoke software. Incremental release imposes a need for rigorous prioritisation and resource estimation for the requirements to select the best subset of requirements for each release. Similarly, requirements elicitation is usually easier where there is a single customer than where there are either only potential customers (for a new product) or thousands of existing but heterogeneous existing customers (for a new release of an existing product).
- The nature of the application. Software requirements engineering means different things to (for example) information systems and embedded software. In the former case, a software development organisation is usually the lead or the only contractor. The scale and complexity of information systems projects can cause enormous difficulties, but at least the software developer is involved from initial concept right through to hand-over. For embedded software the software developer may be a subcontractor responsible for a single subsystem and have no direct involvement with eliciting the user requirements, partitioning the requirements to subsystems or defining acceptance tests. The requirements will simply be allocated to the software subsystem by a process that is opaque to the software contractor. A complex customer / main contractor / subcontractor project hierarchy is just one of many factors that can greatly complicate resolution of any problems that emerge as the software contractor analyses the allocated requirements.
This latter point introduces one of the most controversial aspects of software requirements engineering: the question of ownership. Is requirements engineering a task of software engineering or of systems engineering? In the case of embedded software it is clear that it is a systems engineering task to elicit the user requirements, derive a system architecture composed of appropriate technologies, and allocate the requirements accordingly. In the case of information systems, it is less clear but broadly the same activities have to be performed despite the absence of non-IT technologies. These similarities are likely to become more apparent if, as seems likely, economic imperatives force increasing use of component architectures. Here, solutions are composed of commercial off-the-shelf (COTS) software components integrated with relatively small amounts of bespoke software. The requirements engineering problem then becomes one of finding a best match between the elicited user requirements and the advertised properties of the COTS components.
Because of ambiguity about 'ownership' of the discipline and the artificiality in distinguishing between system and software requirements, we henceforth omit the word 'software' and talk about 'requirements engineering' except where it is helpful to make a distinction.
This is also reflected by the coverage of requirements engineering by standards. Requirements documentation is the only aspect of requirements engineering that is covered by dedicated standards. Wider requirements engineering issues tend to be covered only as activities of software engineering or systems engineering. Current process improvement and quality standards offer only limited coverage of requirements engineering issues despite its place at the root of software quality problems.
There are broadly two common ways of decomposing requirements engineering knowledge. The first is exemplified by the European Space Agency's software engineering standard PSS-05. Here, the knowledge area is decomposed at the first level into the 2 topics of user requirements definition and software requirements specification. This is broadly what we described in section 2 and is a useful starting point (which is why we described it). However, as a basis for the knowledge area breakdown it confuses process activities and products with knowledge and practice.
Process knowledge is crucial but we prefer to separate out process issues from knowledge. The knowledge area topics correspond approximately to process tasks but they tend to be enacted concurrently and iteratively rather than in series as is implied by a breakdown based on PSS-05's model. The breakdown we have chosen is orthogonal to PSS-05. It comprises 5 topics:
- The requirements engineering process.
- Requirements elicitation.
- Requirements analysis.
- Requirements validation.
- Requirements management.
This section is concerned with the requirements engineering process to orient the remaining 4 topics and show how requirements engineering dovetails with the overall software life-cycle. It also deals with contractual and project organisation issues. The topic can be considered to comprise 4 subtopics.
This subtopic is concerned with generic process models that show that the requirements process:
- is not a discrete front-end activity of the software life-cycle but rather a process that is initiated at the beginning of a project but continues to operate throughout the life-cycle;
- is connected to the overall software process by the life-cycle of requirements and the involvement of the requirements engineer;
- will need to be tailored to the organisation and project context.
In particular, the subtopic provides the context that allows the software engineer to understand how requirements elicitation, analysis, validation and management fit together.
This subtopic is concerned with the roles of the people who enact the requirements engineering process. The key people are the system stakeholders (users and other people who have a stake in the software) and the requirements engineer. The software engineering roles of (for example) system architect, tester and quality assurance are also described with reference to requirements engineering.
This subtopic covers the resources required and consumed by the requirements engineering process. This includes human resources, training requirements, methods and tools.
This subtopic is concerned with quality. It covers the key role requirements engineering plays in terms of the cost, timeliness and customer satisfaction of software products. The subtopic orients the requirements engineering process with quality standards and process improvement models for software and systems.
This topic covers what is sometimes termed 'requirements capture', 'requirements discovery' or 'requirements acquisition'. It is concerned with where requirements come from and how they can be collected by the requirements engineer. Requirements elicitation is the first stage in building an understanding of the problem the software is required to solve. It is fundamentally a human activity and is where the stakeholders are identified and relationships established between the development team (usually in the form of the requirements engineer) and the customer. There are 2 main subtopics.
In a typical system, there will be many sources of requirements and it is essential that all potential sources are identified and evaluated for their impact on the system. This subtopic is concerned with different requirements sources and frameworks for managing them. The main points are:
- Goals. The term 'Goal' (sometimes called 'business concern' or 'critical success factor') refers to the overall, high-level objectives of the system. Goals provide the motivation for a system but are often vaguely formulated. Requirements engineers need to pay particular attention to assessing the impact and feasibility of the goals.
- Domain knowledge. The requirements engineer needs to acquire or to have available knowledge about the application domain. This enables them to infer tacit knowledge that the stakeholders don't articulate, inform the trade-offs that will be necessary between conflicting requirements and sometimes to act as a 'user' champion.
- System stakeholders. Many systems have proven unsatisfactory because they have stressed the requirements for one group of stakeholders at the expense of others. Hence, systems are delivered that are hard to use or which subvert the cultural or political structures of the customer organisation. This subtopic is designed to alert the requirements engineer to the need to identify, represent and manage the 'viewpoints' of many different types of stakeholder.
- The operational environment. Requirements will be derived from the environment in which the software will execute. These may be, for example, timing constraints in a real-time system or interoperability constraints in an office environment. These must be actively sought because they can greatly affect system feasibility and cost.
- The organisational environment. Many systems are required to support a business process and this may be conditioned by the structure, culture and internal politics of the organisation. The requirements engineer needs to be sensitive to these since, in general, new software systems should not force unplanned change to the business process.
When the requirements sources have been identified the requirements engineer can start eliciting requirements from them. This subtopic concentrates on techniques for getting human stakeholders to articulate their requirements. This is a very difficult area and the requirements engineer needs to be sensitised to the fact that (for example) users may have difficulty describing their tasks, may leave important information unstated, or may be unwilling or unable to cooperate. It is particularly important to understand that elicitation is not a passive activity and that even if cooperative and articulate stakeholders are available, the requirements engineer has to work hard to elicit the right information. Common techniques are:
- Interviews. Interviews are a 'traditional' means of eliciting requirements. It is important to understand the advantages and limitations of interviews and how they should be conducted.
- Observation. The importance of systems' context within the organisational environment has led to the adaptation of observational techniques for requirements elicitation whereby the requirements engineer learns about users' tasks by immersing themselves in the environment and observing how users interact with their systems and each other. These techniques are relatively new and expensive but are instructive because they illustrate that many user tasks and business processes are too subtle and complex for their actors to describe easily.
- Scenarios. Scenarios are valuable for providing context to the elicitation of users' requirements. They allow the requirements engineer to provide a framework for questions about users' tasks by permitting 'what if?' and 'how is this done?' questions to be asked. There is a link to conceptual modeling because recent modeling notations have attempted to integrate scenario notations with object-oriented analysis techniques.
- Prototypes. Prototypes are a valuable tool for clarifying unclear requirements. They can act in a similar way to scenarios by providing a context within which users better understand what information they need to provide. There are a range of prototyping techniques which range from paper mockups of screen designs to beta-test versions of software products. There is a strong overlap with the use of prototypes for requirements validation.
This subtopic is concerned with the process of analysing requirements to:
- detect and resolve conflicts between requirements;
- discover the bounds of the system and how it must interact with its environment;
- elaborate user requirements to software requirements.
The traditional view of requirements analysis was to reduce it to conceptual modeling using one of a number of analysis methods such as SADT or OOA. While conceptual modeling is important, we include the classification of requirements to help inform trade-offs between requirements (requirements classification), and the process of establishing these trade-offs (requirements negotiation).
4.3.1 Requirements classification
Requirements can be classified on a number of dimensions. Common useful classifications include:
- As capabilities or constraints. Distinguishing between capabilities and constraints can be hard but is worthwhile, particularly where the software's 'qualities' (in safety-related system, for example) are especially important. It is common to sub-classify constraints as, for example, performance, security, reliability, maintainability, etc. requirements. Many such requirements imply the need for special metrics to specify and validate the requirements and have implications for the resources that need to be allocated to testing. It is useful to perform this classification early in the requirements engineering process to help understand the full implication of user requirements.
- According to priority. In general, it will be impossible to implement every user requirement because of mutual incompatibilities between requirements and because of insufficient resources. These conflicting demands have to be traded-off and one of the most important items of information needed to do this is the requirements' priorities. In practice, it can be hard to do this and for many organisations a 3-level classification such as mandatory/highly desirable/desirable provides a crude but workable scheme.
- According to the requirements' cost/impact. This classification complements that of priority since if the cost of implementing a mandatory requirement is unaffordable, this will reveal a serious problem with the stakeholder's expectations. The impact of changing a requirement is also important since it strongly affects the cost of doing so.
- According to their scope. The scope of a requirement can also affect its priority and cost. Requirements with global scope tend to be costly and of high priority; responsibility for their satisfaction cannot be allocated to a single component of the architecture.
- According to volatility/stability. Some requirements will change during the life-cycle of the software and even during the development process itself. It is sometimes useful if some estimate of the likelihood of a requirement changing can be made. For example, in a banking application, requirements for functions to calculate and credit interest to customers' accounts are likely to be more stable than a requirement to support a particular kind of tax-free account. The former reflect a fundamental feature of the banking domain (that accounts can earn interest), while the latter may be rendered obsolete by a change to government legislation. Flagging requirements that may be volatile can help the software engineer establish a design that is more tolerant of change.
- According to whether they are product or process requirements.
Other classifications may be appropriate, depending upon the development organisation's normal practice and the application itself. There is a strong overlap between requirements classification and requirements attributes.
4.3.2 Conceptual modeling
The development of models of the problem is fundamental to requirements analysis. The purpose is to aid understanding of the problem rather than to initiate design of the solution. However, the boundary between requirements engineering and design is often a blurred one in practice and the requirements engineer may find themselves unavoidably having to model aspects of the solution. There are several kinds of models that can be developed. These include data and control flows, state models, event traces, user interactions, object models and many others. The factors that influence the choice of model include:
- The nature of the problem. Some types of application demand that certain aspects are analysed particularly rigorously. For example, control flow and state models are likely to be more important for real-time systems than for an information systems.
- The expertise of the requirements engineer. It is generally better to adopt a modeling notation or method that the requirements engineer has experience with.
- The process requirements of the customer. Customers may impose a particular notation or method on the requirements engineer. This can conflict with the last factor.
- The availability of methods and tools. Notations or methods that are poorly supported by training and tools may not reach widespread acceptance even if they are suited to particular types of problem.
Note that in almost all cases, it is useful to start by building a model of the 'system boundary'. This is crucial to understanding the system's context in its operational environment and identify its interfaces to the environment.
The issue of modeling is tightly coupled with that of methods. For practical purposes, a method is a notation (or set of notations) supported by a process that guides the application of the notations. Methods and notations come and go in fashion. Object-oriented notations are currently in vogue (especially UML) but the issue of what is the 'best' notation is seldom clear. There is little empirical evidence to support claims for the superiority of one notation over another.
Formal modeling using notations based upon discrete mathematics and which are tractable to logical reasoning have made an impact in some specialised domains. These may be imposed by customers or standards or may offer compelling advantages to the analysis of certain critical functions or components.
This topic does not seek to 'teach' a particular modeling style or notation but rather to provide guidance on the purpose and intent of modeling.
4.3.3 Requirements negotiation
Another name commonly used for this subtopic is 'conflict resolution'. It is concerned with resolving problems with requirements where conflicts occur; between two stakeholders requiring mutually incompatible features, or between requirements and resources or between capabilities and constraints, for example. In most cases, it is unwise for the requirements to make a unilateral decision so it is necessary to consult with the stakeholder(s) to reach a consensus on an appropriate trade-off. It is often important for contractual reasons that such decisions are traceable back to the customer. We have classified this as a requirements analysis topic because problems emerge as the result of analysis. However, a strong case can also be made for counting it as part of requirements validation.
It is normal for there to be one or more formally scheduled points in the requirements engineering process where the requirements are validated. The aim is to pick up any problems before resources are committed to addressing the requirements.
One of the key functions of requirements documents is the validation of their contents. Validation is concerned with checking the documents for omissions, conflicts and ambiguities and for ensuring that the requirements follow prescribed quality standards. The requirements should be necessary and sufficient and should be described in a way that leaves as little room as possible for misinterpretation. There are four important subtopics.
4.4.1 The conduct of requirements reviews
Perhaps the most common means of validation is by the use of formal reviews of the requirements document(s). A group of reviewers is constituted with a brief to look for errors, mistaken assumptions, lack of clarity and deviation from standard practice. The composition of the group that conducts the review is important (at least one representative of the customer should be included for a customer-driven project, for example) and it may help to provide guidance on what to look for in the form of checklists.
Reviews may be constituted on completion of the user requirements definition document, the software requirements specification document, the baseline specification for a new release, etc.
4.4.2 Prototyping
Prototyping is commonly employed for validating the requirements engineer's interpretation of the user requirements, as well as for eliciting new requirements. As with elicitation, there is a range of elicitation techniques and a number of points in the process when prototype validation may be appropriate. The advantage of prototypes is that they can make it easier to interpret the requirements engineer's assumptions and give useful feedback on why they are wrong. For example, the dynamic behaviour of a user interface can be better understood through an animated prototype than through textual description or graphical models. There are also disadvantages, however. These include the danger of users' attention being distracted from the core underlying functionality by cosmetic issues or quality problems with the prototype. They may also be costly to develop although if they avoid the wastage of resources caused by trying to satisfy erroneous requirements, their cost can be more easily justified.
4.4.3 Model validation
The quality of the models developed during analysis should be validated. For example, in object models, it is useful to perform a static analysis to verify that communication paths exist between objects that, in the stakeholder's domain, exchange data. If formal specification notations are used, it is possible to use formal reasoning to prove properties of the specification (e.g. completeness).
4.4.4 Acceptance tests
An essential property of a user requirement is that it should be possible to verify that the finished product satisfies the requirement. Requirements that can't be verified are really just 'wishes'. An important task is therefore planning how to verify each requirement. In most cases, this is done by designing acceptance tests. One of the most important requirements quality attributes to be checked by requirements validation is the existence of adequate acceptance tests.
4.5 Requirements management
Requirements management is an activity that should span the whole software life-cycle. It is fundamentally about change management and the maintenance of the requirements in a state that accurately mirrors the software to be, or that has been, built.
There are 4 main topics concerned with requirements management.
4.5.1 Change management
Change management is central to the management of requirements. This subtopic is concerned with the role of change management, the procedures that need to be in place and the analysis that should be applied to proposed changes.
4.5.2 Requirements attributes
Requirements should consist not only of a specification of what is required, but also of ancillary information that helps manage and interpret the requirements. This should include the various classifications attached to the requirement and the verification method or acceptance test plan. It may also include additional information such as a summary rationale for each requirement, the source of each requirement and a change history. The most fundamental requirements attribute, however, is an identifier that allows the requirements to be uniquely and unambiguously identified. A naming scheme for generating these IDs is an essential feature of a quality system for a requirements engineering process.
4.5.3 Requirements tracing
Requirements tracing is concerned with recovering the source of requirements and predicting the effects of requirements. Tracing is fundamental to performing impact analysis when requirements change. A requirement should be traceable backwards to the requirements and stakeholders that motivated it (from a software requirement back to the user requirement(s) that it helps satisfy, for example). Conversely, a requirement should be traceable forwards into requirements and design entities that satisfy it (for example, from a user requirement into the software requirements that have been elaborated from it and on into the code modules that implement it).
The requirements trace for a typical project will form a complex directed acyclic graph (DAG) of requirements. In the past, development organisations either had to write bespoke tools or manage it manually. This made tracing a short-term overhead on a project and vulnerable to expediency when resources were short. In most cases, this resulted in it either not being done at all or being performed poorly. The availability of modern requirements management tools has improved this situation and the importance of tracing (and requirements management in general) is starting to make an impact in software quality.
4.5.4 Requirements documentation
This subtopic is concerned with the role and readership of requirements documentation, with documentation standards and with the establishment of good practice.
Requirements documents are the normal medium for recording and communicating requirements. It is good practice to record the user requirements and the software requirements in different documents. The user requirements definition document organises the user requirements and makes them available to the stakeholders for validation. Once validated, the user requirements can be elaborated and recorded in the software requirements specification document (SRS). Once this has been validated, it serves as the basis for subsequent development.
Requirements documents need to be subjected to version control as requirements change. Modern requirements management tools allow requirements to be documented in electronic form along with their associated attributes and traceability links. Such tools usually include document generators that allow requirements documents to be generated in defined formats so, for example, the software requirements can be used to (semi-) automatically generate an SRS that conforms to IEEE std 830.