Analyze

Introduction

In this module you will study activities involved in the second phase of the risk management paradigm. You will learn and understand three analysis activities: Evaluate attributes of risk , Classify risks, and Prioritize (ranking) risks. The activities associated with filling in the risk information sheet.

Content

1. Analyze Objectives
2. Analyze Introduction
3. Evaluating
4. Impact
5. Timeframe
6. Probability
7. Classify
8. Prioritize
9. References

Analyze Objectives:

• Learn and understand three analysis activities:
  1. Evaluate attributes of risk
  2. Classify risks
  3. Prioritize (ranking) risks

• Learn about and be able to perform key methods and use key tools.
• What the risk analyst does with the risks list once the team has compiled the set of risks.

Analyze Introduction

Lets go back and visualize Roger L Van Scoy's paradigm [Van Scoy 92, p.9] and what he said. To refresh you memory, Van Scoy said:

"Risk analysis is the next element in the risk management paradigm. Risk analysis is the conversion of risk data into risk management information. Each risk must be understood sufficiently to allow a manager to make decisions. Risk analysis sifts the known risks, and places the information in the hands of the decision maker. Analysis provides the information that allows managers to work on the right risks."

The paradigm illustrates a set of functions that are identified as continuous activities throughout the life cycle of a project. The Analyze phase is highlighted in Figure Analyze Risk Management Paradigm.

Figure Analyze Risk Management Paradigm

The purpose of Analyze is to convert the Statement of risks and list of risks into decision-making information. Analysis is a process of examining the risks and context statements in detail to determine the extent of the risks, how the risks relate to each other independently or in a set, and which risks are the most important. Analyzing risks has three basic activities: evaluating the attributes of the risks (impact, probability, and timeframe), classifying the risks, and prioritizing or ranking the risks.

The Figure Analyze Process shows the inputs and outputs of the Analyze function.

Figure Analyze Process

Inputs:

• List of Risks
• Risk Information Sheets
• • Statement of risk
  • Context

• Risk Information Sheet
• • Statement of risk
  • Context
  • Impact
  • Probability
  • Timeframe
  • Classification
  • Rank
• Classification
• Master List of risks

The following are the description of inputs and outputs to the analyze function. The list of risks contains all the statements of risk that need to be analyzed. Prior to analysis, the risk information sheet for each risk contains the statement of risk and its supporting context. After analysis, values for impact, probability, timeframe, class, and rank are added to the risk information sheet (statement of risk, supporting context) for each risk. Classification organizes risks into groups having some common basis. The organization may come from a predefined structure or from a self-organized structure. This list is an organization of the risks according to its classification. The master list of risks contains all risks that have been identified and the priority ranking of the top N risks.Since the Risk Information Sheet is a key configuration-tracking artifact, it will be described first. The Risk Information Sheet has been started in the Identify phase and contains the risk statement and its supporting context statement. During the analyze phase, the risk analyst adds to the risk information sheet for each risk values for impact, probability, timeframe, class, and rank. Figure Risk Information Sheet for Analyze is an example of the areas to be filled in on the risk information sheet for analyze.

Figure Risk Information Sheet for Analyze

The shaded areas indicate the five fields that the risk analyst fills out during Analyze phase of the paradigm.

1. Priority: The priority ranking of the risk
2. Probability: The likelihood of occurrence-exact value depends on the level of analysis
3. Impact: The degree of impact which will depend on the exact value depends on the level of analysis
4. Timeframe: The timeframe in which action is needed
5. Class: The classification of the risk (could be more than one value); the class or group the risk belongs to

The three Analyze activities (evaluate, classify and prioritize) will now be expanded upon.

Evaluating

The objective of evaluating the attributes is to gain a better understanding of the risk by determining the expected impact, probability, and timeframe of the risk. The Figure Evaluate shows the inputs and outputs for evaluating the attributes of risks

Evaluating is the first step of the three steps in the analyze phase. Evaluating provides a better understanding of the risk by qualifying the expected impact, probability, and timeframe of a risk. Evaluating the attributes of a risk involves establishing the current values for:

• Impact: the loss or negative affect on the project should the risk occur
• Probability: the likelihood that the risk will occur
• Timeframe: the period when the risk analyst must take action in order to mitigate the risk.

Impact

Risks should be evaluated at a level of analysis that is sufficient to determine the relative importance, for planning cost-effective mitigation strategies, and to support tracking. Therefore, individual risks can be analyzed and managed at various levels of detail at different points in time.

As an example a high impact, high probability risk may require a more detailed level of analysis to plan a mitigation strategy. In contrast, simply knowing a risk is not likely to occur (low probability) and will have an insignificant impact (low impact) if it does occur maybe all the risk analyst needs to do is to decide how to deal with the risk.

Table Levels of Analysis lists some ranges of attribute values for a risk at various levels of analysis. It is representative of many possible levels. There is a wide range of levels possible between the binary level and the n-level. There could be four levels, ten levels, etc. It is also possible to have a combination of levels for attributes of a given risk.

Table Levels of Analysis

The Table Levels of Risk Exposure summarizes the various values of the risk exposure associated with the range of Levels of Analysis.

Table Levels of Risk Exposure

Individual risks can be analyzed and managed at various levels of detail at different points in time. For example this is the first time the risk analysis team has identified the risks on a project and they came up with 100 risks. They could do a detailed analysis on all 100 risks but this would be very time consuming and many details might be lost in the effort. The team might perform a binary level analysis to quickly sort out the risks that could potentially be catastrophic or critical to the project. For some of the risks the team may want to do a more detailed analysis. Therefore, various levels of analysis could be used to filter a large number of risks into descending order levels. The purpose is to show that risks should be evaluated at a level of analysis that is sufficient:

1. To determine the relative importance
2. For planning cost-effective mitigation strategies
3. To support tracking

1. Impact: catastrophic, critical, marginal
2. Probability: very likely, probable, improbable
3. Timeframe: near-term, mid-term, far-term

The attribute values for each risk are determined based on specific criteria. The results will be more useful if the project team defines the criteria for the attribute values that make sense to the project. These need to be defined in the Risk Management Plan. One person's critical could be another person's marginal. The definitions need to be set by the project team at the beginning of the project and maintained throughout the life cycle. The more specific the criteria, the easier it will be for risk team participants to evaluate the risks. Vague criteria are open to interpretation. Project risk team personnel should apply the criteria consistently.

The Table Air Force Summary shows the AFSC/AFLC Pamphlet 800-45 [Air Force 88, p. 153] example of risk exposure.

Table Air Force Summary

The risk team should be aware of performing multiplication on the ordinal scale values, if the impact and probability have been evaluated qualitatively using ordinal numbers. The individual scale values provide information on the impact and probability of the risk. Multiplying these ordinal values to obtain risk exposure provides information that if not careful thought out could be misinterpreted.

The following Table Risk Exposure and Ordinal Numbers shows ordinal values applied to the Air Force impact and probability values as example as well as the combined values for risk exposure. Consider a risk, X, which is evaluated as critical (3) and frequent (4). This risk has a high-risk exposure, which is calculated as the product of impact and probability, which is 12. Consider a second risk, Y, which is evaluated as critical (3) and improbable (2). The risk exposure for Y is 6, a moderate risk exposure. With ordinal numbers all one can say is that risk X has a higher risk exposure than risk Y. It is tempting to say that risk X has twice the risk exposure than risk Y. With ordinal numbers one cannot say how much higher the risk exposure for risk X is than risk Y. The danger comes when one applys more meaning to numbers than the numbers support.

Table Risk Exposure and Ordinal Numbers

Another example shown in Table NASA Safety Impact Definitions are the definitions used by safety analysts/engineers at NASA

Table NASA Safety Impact Definitions

These terms are used by safety analysts/engineers at NASA to help categorize potential hazardous risks. When determining if a particular risk has safety implications, it is best to use what the safety analyst looks at to determine safety risk. The risk team should work with a safety analyst/engineer and report any risks that have the potential for injury or mission damage. In the safety engineering area, any possibility that a catastrophic or critical failure may occur is deemed worthy of writing a hazard report about. That hazard report may explain why it is highly unlikely for that particular event to occur and will document what safeguards are in place to prevent the occurrence of such an event. It is still a hazard even if it is remotely likely to occur and several safeguards are in place.

An example with quantifiable numbers associated with the impact is shown in Table Quantifiable Impact Definitions.

Table Quantifiable Impact Definitions

Again remember that the risk manager/project manager must do the definitions at the beginning of the project and the definitions must be included in the risk management plan. If the impact is negligible then it may not be necessary to carry the activity as a risk, but the risk identified must first be evaluated to determine its importance to the project.

Timeframe

The Timeframe is the time when action to mitigate a risk must be taken. This attribute is added to the Risk Information Sheet. The following are examples of term and quantifiable values that could be used. These relevant time values are use to fill in the example project timeframes used for IR-SIP.

Example Timeframe Definitions:

• A risk is near-term if the project must take action in the next 90 days.
• A risk is mid-term if the project must take action in the next 90-180 days.
• A risk is far-term if the project need not take action in the next 180 days.

The three key words are near, mid, and far term with the associated number of suggested days. The days are flexible depending on the size of the project.

Probability

The Probability definition is the likelihood that the risk will occur. This attribute is added to the Risk Information Sheet. Each project team will have to define and agree on the ranges as applicable for there levels of risk.

The following are example Probability Definitions:

• A risk is very likely if there is a >70% probability that it will occur.
• A risk is probable if there is a 30-70% probability that it will occur.
• A risk is improbable if there is a <30% probability that it will occur.

Interactive Tri-Level Attribute Evaluation TBD xxxx

Consider the purpose of the evaluation effort. The time and resources required for the evaluation must be balanced against the value of the added level of information. For example, initially the team may choose a binary level of analysis to sort through a large number of risks. Then the team may decide that for a few of the more important risks they would like to revisit the evaluation with a more refined measure of the attributes. Choosing a level of analysis depends on a number of factors, such as:

• What fits in your organization?
• What does a customer or policy prescribe?
• What is sufficient for planning a mitigation strategy for an individual risk?

Classify

Classify is the second step in the Analyze phase.There are several ways to classify or group risks. The ultimate purpose of classification is to understand the nature of the risks facing the project and to group any related risks in order to build more cost-effective mitigation plans. The process of classifying risks may reveal that two or more risks are equivalent. The statements of risk and context statement indicate that the subject of these risks is the same. Equivalent risks are therefore duplicate statements of the same risk and should be combined into one risk or a set of risks. Classifying risks involves grouping risks based on shared characteristics. The groups or classes show relationships among the risks. Classification helps to identify duplicate risks and supports simplifying the list of risks. The objective of classifying risks is to look at a set of risks and how those risks relate to each other within a given structure. The classes or groups of risks provide a different perspective when planning risks.

The Diagram Classify shows the inputs and outputs for classifying risks.

Diagram Classify

Within the Continuous Risk Management approach, risks are classified using two conceptual perspectives as listed in the Table Classification Perspectives.

Table Classification Perspectives

Two criteria for grouping risks are by classifying by source or impact. When classifying risks using the predefined structure, the criterion chosen will affect the outcome of groups of risks. The two criteria for grouping risks are:

• By source: Risks are grouped based on the same source or root cause. This will show the major sources of risk to the project.
• By impact: Risks are grouped based on where or how the impact will be felt by the project. This shows the project the major product areas that will be impacted by the risks.

Classification by impact can occur at several levels. Risks may be classified by their impact on technical work, budget, or schedule. This high level classification can show a manager, which risks the customer may see and which risks are primarily internal to the project. A useful classification for planning might look at a more detailed view of where the impact will be felt such as a product subsystem.

There are several classification uses or ways to classify or group risks. The ultimate purpose of classification is to understand the risks the project faces and group related risks to help build more cost-effective mitigation plans. Multiple views may provide insight into how best to deal with the risks in planning. It is important to maintain the classification structure during planning. The classification is not helpful if it is not used consistently in planning. If the structure is changed, the risk teak should reclassify all the risks. Some examples the team could use to classify are:

Cost Organizational

Schedule Management

Technical Political

Safety Supportability

The first time project members identify risks; they may come up with a large number. This is called multiple classifications. Initially, they may classify according-to the source of risk to understand the global risk picture. For example which risks are resulting from requirements instability? However, mitigating the risks may best be done by a different classification based on the group or person assigned to deal with the risk or what other risks affect the same area. The computer engineer would be assigned the risks that are affecting the compiler performance. Both views provide valuable information to the project.

There are no specific rules for selecting a classification scheme. The project team should consider what would help during the planning process. The team should use a database to store, sort, and merge multiple classification information to make it manageable.

The result of a classification may be shown as a Bar Graph in Figure Classification Bar Graph. A classification bar graph is a graphic display of the groups in a classification and the number of risks in each group.

Figure Classification Bar Graph

The bar graph indicates the number of risks that were classified, based on the source of the risk, into each taxonomy element of the software development risk taxonomy class/element/attribute structure [Carr 93].

Duplicate risks should be combined or put into sets of risks. The process of classifying risks may reveal that two or more risks have equivalent risk and context statements. This will indicate that the subject of these risks is the same. Equivalent risks are therefore duplicate statements of the same risk and should be combined into one risk.

When the team merges some classes into sets they should write a new summary risk statement that consolidates all the risks into one. Then assign the summary risk statement a new ID number. The individual risk statements (and their original ID numbers) become part of the context statement, and the entire context from each risk is merged. The project doesn't want to lose captured risk information. The attribute evaluation information for impact, probability, and timeframe should reflect the worst case of the individual risks.

Table Consolidating Risk shows that three risks ID 1, 16 and 17 are very similar and can be merged into a new summary risk statement and given a new ID 101. A numbering scheme for the summary risk statement should be developed that works for the project and new number should differentiate consolidated risks. In the example a 3-digit number (ID 101) is used.

Table Consolidating Risk

Table Methods and Tools summarize the methods and tools for classifying risks.

Table Methods and Tools

Prioritize

The third and final step in the Analysis phase, is the risk team's activity of prioritizing (ranking) the risks. The purpose is to sort through a large number of risks and determine which are most important and to separate out which risks should be dealt with first when allocating resources. This involves partitioning risks or groups of risks based on the "vital few" sense and ranking risks or sets of risks based on consistently applying an established set of criteria. No project has unlimited resources with which to mitigate risks. Thus, it is essential to determine consistently and efficiently which risks are most important and then to focus those limited resources on mitigating risks. Figure Prioritize shows the inputs and outputs for prioritizing risks.

Figure Prioritize

Conditions and priorities will change during a project, and this natural evolution can affect the important risks to a project. Risk analysis must be a continuous process throughout the project life cycle. Analysis requires open communication so that prioritization and evaluation is accomplished using all known information. A forward-looking view enables risk team personnel to consider the long-range impacts of risks. Prioritizing (ranking) risks involves partitioning risks or groups of risks based on the Pareto (Pa-ray'-to) "vital few" sense [Juran 89] and ranking the risks or sets of risks based upon a criterion or set of criteria as appropriate. The risks team will be spending most of their time and effort on. The vital few or 80/20 rule for risk means that the team will spend 80% of their resources on the top 20% of the risks.

Vilfredo Pareto (1848-1923) was an Italian economist and sociologist who posited that 80% of the world's wealth and power resided with 20% of the population and that, if the wealth and power were evenly distributed over the entire population, 80% of the wealth and power would eventually return to the hands of the 20%.

Sets of risks may be prioritized along with singular risk statements because a project's risks are dealt with at various levels of complexity. One singular risk statement may warrant being dealt with by itself due to the nature of the risk, while another may best be dealt with by grouping them with other risks that are related. In other words, sometimes a risk is only seen when all the component pieces (i.e., smaller, related risks) are put together. It is not uncommon to deal with both single risks and sets of risks at the same time. The objective of prioritizing risks is to separate out which risks should be dealt with first (the vital few risks) when allocating resources.

Ranking the top N risks or groups of risks involves taking the list of top N risks and ordering these based upon a criterion or set of criteria into a rank ordered list. Selecting the "N" risks involves using the attribute (impact, probability, and timeframe) values that were evaluated. The Pareto Top N risks are the risks that will be prioritized based on the criteria selected for the project as well as the attributes of impact, probability, and timeframe. Remember that risk attributes are good guidelines, but are not an exact science. The list of risks contains single risks as well as sets of risks. The Diagram Top N Risks shows a top N list made up of single risks and sets of risks.

Diagram Top N Risks

The Table Pareto Top 20% is an example of the Pareto Top 20% using the risks in the IR-SIP.

Table Pareto Top 20%

The risks are sorted from highest to lowest based on the values for probability, impact, and timeframe that were the result of the Tri-Level Attribute Evaluation method. For the IR-SIP project, a total of 30 risks were identified (note: not all risks are shown). The 10% and 20% break points are indicated. At the 10% break point, not all of the high-risk exposure risks are included. The 20% break point hits after the high-risk exposure values. In the IR-SIP Project Manager Jerry Johnstone decided to draw the line at moderate risk exposure with near-term timeframe but has special concerns about risk ID 7 and he therefore decides to draw the line at 26% and include risk ID 7. The criteria may change over the life of the project. Prioritization of risks will also change over the life of the project. As time progresses risks will emerge, change, and go away. Risks will be reprioritized as part of continuously managing risks on the project.

The Potential Top N is another method that selects the most important risks to a project base on the individual knowledge of the participants in the project. The potential top N is generated by sequentially selecting risks from each individual top 5 lists in rounds until all risks are selected. The result is a non-ordered list of important risks to the project. When the method is being used during the Analyze activity or any other group activity, a individual top 5 lists per each participant.

The criterion or set of criteria used to rank the risks is chosen based on what's most important to the project. For example:

• Meeting the timing requirement for a servo motor
• Meeting a schedule for a major milestones
• Keeping cost within the budget

The Potential Top N selection process is shown in the Figure Potential Top N Selection.

Figure Top N Selection

While the project-wide Pareto "vital few" can be managed at the highest levels, all of the other risks can be managed within the departments or teams of the organization most suited to effectively manage those risks. In the Plan phase these risks will be delegated to the appropriate level of management.

The Multivoting method is a general voting method. It can be used to conduct a straw poll or select the most important items from a list with limited discussion and limited difficulty. For a large number of items, a series of votes is used to reduce the list to a workable number. Each participant in the process votes on the items in the list.

The method is to selecting the number of votes to be used depends on the number of items on the list. A general rule of thumb is to allow evaluators votes to be equal to one-third the number of items on the list. The Figure Multivoting is an example of a tally form used to combine all evaluators' votes. There are 5 evaluators; there are a total of 12 risks (labeled A - H) so each participant gets 3 (or 4) weighted votes. This example used 3 votes. Selecting the voting criteria depends on the project objectives and constraints. The evaluators must decide what criteria are appropriate to use for ranking based on what's important to the project. Some examples are:

• Which items have a significant impact?
• Which items are more likely to occur?
• Which items have a greater impact on performance?

After everyone votes, the totals for each risk are tallied, thus in this example, ranking the risks with H the highest B next then A C J. The other risks did not get any votes.

The Figure Multivoting

The following is an exercise for you to perform using the partial list of 10 risks given below. These risks were the ones selected using the Pareto Top 20% method. Identify those risks you feel are the three most critical to the project based on the prioritization criteria:

• Must meet the schedule
• Can't delete any of the technical or performance requirements
• Must keep to the budget

Vote for the three risks that are most important to the project based on the prioritization criteria. Give the most important risk 3 points, the next most important risk 2 points, and give the third most important risk 1 point.

Table List of 10 Risks

Comparison risk ranking (CRR) is the last method examined in which risks are ranked by comparing the risks, two at a time, to an established criterion or set of criteria (stated in the form of a question). Each risk is compared to every other risk. Each participant in the process casts a vote in each comparison. An individual or a group can do comparison risk ranking. If performed by a group of three or more, one person should be the facilitator and recorder (but he or she could still participate or contribute).

In you opinion as a risk evaluator which risk (ID 2 or 5) is more important and may cause the project t

• Not meet the schedule
• Delete a technical or performance requirement
• Not keep to the budget

(ID 2) Commercial parts are being selected for space flight applications and their suitability to meet environmental conditions is unknown; these parts may fail to operate on-orbit within the environment window, leading to system level failures. Also, environmental testing of these parts can be expensive and cause schedule delays.

OR

(ID 5) Lack of a thorough hardware test program; mission failure due to environmental conditions not tested.

The Comparison Risk Ranking method compares two risks at a time with respect to the project criteria. There is an explicit comparison of every risk with every other risk and time for the participants to discuss each comparison and share their knowledge with each other prior to voting. The purpose of the discussion is to point out that prioritizing risks also provides an opportunity for communication within the project. Even with defined criteria, individuals will have their own background information that will affect how they prioritize. Multivoting is a good way to conduct a straw poll and see where individuals' initial rankings are. It's a good way to reduce a large list. Comparison Risk Ranking is an excellent way for project members to share information about risk during the prioritization process for a small number of risks.

The Table Prioritization Methods and Tools summarize the methods and tools for prioritizing risks.

Table Prioritization Methods and Tools

In summary no project has unlimited resources to mitigate risks. The risk team must determine what risks are most important and focus their resources on mitigating those risks.

The way the team determines the top N risks are:

1. Evaluate: for each risk (using the tri-level attribute evaluation method) determined values for:

• Impact
• Probability
• Timeframe

2. Classify: Organized risks into groups using three approaches by:

• Affinity grouping
• Source in the taxonomy
• Impact to the project

3. Prioritize:

• Identified the Pareto top N risks using the attribute (impact, probability, and timeframe values) from the tri-level attribute evaluation methods.
• Then rank ordered the Pareto top N risks based on the criteria selected for the project (using the Multivoting method).

References

[Air Force 88] Air Force Systems Command/Air Force Logistics Command Pamphlet 800-45. Software Risk Abatement, September 30, 1988.

[Boehm 89] Boehm, Barry. IEEE Tutorial on Software Risk Management. New. York: IEEE Computer Society Press, 1989.

[Brassard 89] Brassard, Michael. The Memory Jogger Plus: featuring the seven management and planning tools. Methuen, Ma.: GOAL/QPC, 1989.

[Carr-93] Carr, Marvin;.Konda, Suresh; Monarch, Ira; Ulrich, Carol; & Walker, Clay. Taxonomy -Based Risk Identification (CMU/SEI-93-TR-6, ADA266992). Pittsburgh, Pa.: Software Engineering Institute, Carnegie Mellon University, 1993.

[Juran 89] Juran, J. M. Juran on Leadership for Quality. New York: The Free Press, 1989.