We have been and still are, for more than 22 years now, in the business of providing specialized software telecommunication services. During this time, we delivered most of our projects within schedule, even when some of them were complex and very challenging.
For some of those projects, the big challenge was to accommodate the client’s need to reduce development time, but maintain scope. Under these conditions, our response was tailored as a smart approach that allowed us to develop software in an efficient and effective manner.
We implemented it throughout the development phase and made sure testing was also brought on board.
The R Systems QA team response plan to this challenge was a combination of powerful automation framework, the R Systems-customized JMeter testing framework (for telecommunication environment needs), and effective and efficient ways of doing manual testing.
Embracing risk-based testing as a guide for testing activities
Risk-based testing, or RBT, is a type of software testing that functions as an organizational principle and is used to prioritize software feature and function testing, based on:
- Their risk of failure
- Their importance
- The failure likelihood or impact
As a general idea, the main goal of testing is to uncover defects which may lead to undesirable consequences or potential issues in the long run. The potential for such undesirable event is usually referred to as quality risk, product risk, or quality product risk. Uncertainty implies that the risky event may or may not happen with a percent of probability. Loss, in turn, implies that if a risk becomes a reality, undesired consequences, or more exactly a loss, will occur.
RBT is a test approach or test strategy that takes these risks into account. By identifying and analyzing the risks related to the system, it is possible to make your test effort more efficient and effective:
- Efficient because you tackle first the most critical areas of the system (the earlier the defect is detected the lower is the cost of solving those issues)
- Effective because your time is spent according to the risk rating and mitigation plans
A reduction in cost per quality and faster time to market are more easily achievable with this approach.
This concept, however, is not new for the software development world and is often used in the waterfall software development model.
Our success in using risk-based testing
Our success in using this approach relies on the way we capitalize on our enhanced telecommunication business knowledge and the quality assurance experience of our engineers. Both these factors enable us to create a substantially better risk analysis.
The risk analysis objective is to identify potential issues that could affect the project’s cost or outcome. To perform this, we first conduct a risk identification through which we collect information about the project and then classify it to determine the amount of potential risk.
Regarding risk strategy and risk assessment, we use this information to evaluate risks (cost and probability), and then develop a mitigation plan for of all risks involved. You can see below a sample table for such a risk mitigation plan.
| Risk Priority | Testing extent | Comments |
| 5 | None | Only report bugs observed for this risk item if discovered during other tests. |
| 4 | Opportunistic | Leverage other tests to explore the risk item with minimal effort and only if opportunity presents itself. |
| 3 | Cursory | Run a small number of tests that sample the most interesting conditions for the risk item. |
| 2 | Broad | Run a medium number of tests that exercise many different interesting conditions for the risk item. |
| 1 | Extensive | Run a large number of tests that exercise many combinations and variations of interesting conditions. |
We use testing plans to direct risk management during the software testing activities, so an appropriate testing level is selected based on this risk assessment’s output.
Risk mitigation/avoidance activities avoid risks or minimize their impact by using inspection and/or focus testing on the critical functions to minimize the impact of a failure in a function/feature.
On risk reporting we monitor, based on information obtained from the previous topics, the number of errors found, number of errors per function, classification of errors, number of hours testing per error etc. Risk prediction involves forecasting risks by using the history and knowledge of previously identified risks.
8 advantages of risk-based testing
Risk-based testing helped us achieve better product quality and fulfilled our need to create an efficient and effective way of doing testing work.
- We were able to define more clearly when to stop testing
- Test cases were reduced and focused on the most critical areas
- A smaller number of test cases were needed, due to higher testing efficiency
- Tests were more focused
- Cost was reduced, as important problems were discovered early and preventive activities started sooner
- Overall test goals, strategies, and directions improved
- Testing mainly focused on the most important areas
- Testing was continuously adjusted against problematic areas