This success story details the scenario for using the test automation solutions, specifically the R Systems TOP Testing Suite, for setting the right technology roadmap in view of DevOps and 5G cloud solutions.
Business case
Our end-customer, tier-1 US operator, was looking for a single test tool to perform regression, platform, and performance testing of its subscriber data management (SDM) platforms HSS, SPS, AAA, EIR. R Systems TOP Testing Suite (TTS) was selected based on the recommendation of US technology vendor in 2011.
R Systems solution
Over the years, TTS evolved significantly with various plugins, features and enhancements of which our end customer’s favourite are Diameter, LDAP, Telent/SSH, Radius, HTTP/SOAP. R Systems developed several enhancement features based on the operator’s recommendation and helped it to completely develop all test automation framework using TTS. The client automated approximately 3,000 test cases across all platforms and was able to achieve ~20,000 Diameter TPS using a single thread proving great performance/capacity testing capability.
Some of the technical KPIs of the platform under test – bare metal & virtual HSS – are summarized below:
- Single thread from TTS can now generate ~4,000 threads/sec
- Client will be testing 4 threads with 16,000 threads/sec towards one HSS VNF
- TTS command line can generate more traffic compared to GUI (GUI achieves 20% less threads/sec)
- Average duration of performances tests: 48 hours
In addition to technical performance, the client appreciates the tool for its ease of use, good documentation of features and release notes and solid support services. According to our end customer’s representatives:
“R Systems engineers are very supportive and knowledgeable, they demonstrated professionalism and skills on many occasions. They understand our requirements and deliver expected results. Overall we are very happy customer of R Systems TOP Tool, keep up the good work and we expect the continued support.”
Key tool in the DevOps, cloud and 5G technology roadmap
Looking ahead to future projects in the area of cloud and 5G, R Systems TOP Testing Suite is a key tool in the client’s technology roadmap. CI/CD support, Jenkins integration, J-unit plugin support is recommended and this sets a great roadmap for the DevOps model, while the 5G HTTP2 demo provided built the client’s confidence to pursue the R Systems automated testing tool even for 5G cloud solutions. Also, according to them:
“We would like to continue partnering with R Systems for future DevOps components like centralized logging (Elasticsearch, Logstash, Kibana, ELK code name); centralized statistics (Graphite); centralized deployment and configuration, (Chef); automated testing and regression support.”