Test Execution is one activity which is most difficult to estimate. Most of the time it happens that if testing team find big chunk of bugs in the software then frequent releases/builds are given to testing team due to which testers face time crunch in test execution. This might results into the skipping of *critical* bugs during testing if high priority areas are not well tested.
To avoid these situations, Time-boxing should be achieved. In Software Testing, Time-boxing can be achieved by doing Risk Analysis.
For any software under test, Risk analysis is carried out to decide the depth of testing required to test the software/Functionality.
Steps involved in risk analysis:
1. Form a brainstorming team – This team consists of experienced Development Lead/Architect; Test Lead/Architect; Business Analyst/Product Manager;
2. Gather required documents – Team should gather Specification documents, previous bug reports (if available), Technical design documents, and gather list of functionalities to be tested, etc.
3. Determine Likelihood of failure & Impact of failure for each functionality.
Likelihood is the probability of failure – What is the likelihood that a particular functionality would work incorrectly? Experienced testers and Development Lead/Architect can help in determining the probability of failure.
Impact of failure – What will be the impact to user if the particular functionality failed to work correctly?
Likelihood and impact can be determined by considering the following:
- Where a failure would be most serve (take help from development team)
- Where failures would be most visible (take help from development team)
- Take the help of customer/Business Analyst in understanding what is most important to customer.
- What is most critical to the customers’ business? (Take the help of customer/Business Analyst)
- Areas changed most often. (take help from development team)
- Areas with most problems in the past. (take help from development team)
- Most complex areas, or technically critical. (take help from development team, Business Analyst)
Once likelihood of failure is determined then assign values to all functionalities to be tested.
Assign 3 values to Likelihood – High, Medium & Low
Once Impact of failure is determined then assign values to all functionalities to be tested.
Assign 3 values to Impact – High, Medium & Low
Now we have determined the likelihood and impact of failure for each functionality. Next step is to determine the Priority.
Make sure to Review the assigned Likelihood and Impact for each feature.
4. To determine the Priority, Likelihood and Impact need to be summed up. Let’s assign numeric values to High, Medium & Low.
High = 3; Medium = 2; Low = 1.
So now if Likelihood is High and Impact is Medium; then the priority of feature to be tested is 5.
To the above table, lets add a column for priority which is a sum of Likelihood and Impact.
Team should meet again and review the assigned Priorities.
Let’s sort it to make it easily readable.
So more testing effort is required for Features 1, 3, 5, 6, 9 & 10 than Features 2, 4, 7 & 8.
(Include review process after each step).
Note – Publish the Risk analysis details to the management and other stakeholders. They should be aware of the fact that testing team is not going to test all features completely.
– Happy Testing
References:
1. Advanced Software Testing – Vol. 2 by Rex Black
2. Systematic Software Testing -by Rick D. Craig and Stefan P. Jaskiel
3. So Many Tests, So Little Time by By Johanna Rothman
4. Practical Risk Based Testing: The PRISMA Method by Erik van Veenendaal