The case study details the scenario for using the test automation solutions, specifically the Computaris TOP Testing Suite, for setting the right technology roadmap in view of DevOps and 5G cloud solutions.
Sprint, tier-1 US operator, now part of T-Mobile USA, was looking for a single test tool to perform regression, platform and performance testing of Sprint subscriber data management (SDM) platforms HSS, SPS, AAA, EIR. Computaris TOP Testing Suite (TTS) was selected based on the recommendation of US technology vendor in 2011.
Over the years, TTS evolved significantly with various plugins, features and enhancements of which Sprint’s favourite are Diameter, LDAP, Telent/SSH, Radius, HTTP/SOAP. Computaris developed several enhancement features based on Sprint’s recommendation and helped Sprint to completely develop all test automation framework using TTS. The client automated approximately 3,000 test cases across all platforms and was able to achieve ~20,000 Diameter TPS using a single thread proving great performance/capacity testing capability.
Some of the technical KPIs of the platform under test – bare metal & virtual HSS – are summarized below:
- Single thread from TTS can now generate ~4,000 threads/sec
- Client will be testing 4 threads with 16,000 threads/sec towards one HSS VNF
- TTS command line can generate more traffic compared to GUI (GUI achieves 20% less threads/sec)
- Average duration of performances tests: 48 hours
In addition to technical performance, the client appreciates the tool for its ease of use, good documentation of features and release notes and solid support services. According to Sprint representatives:
“Computaris engineers are very supportive and knowledgeable, they demonstrated professionalism and skills on many occasions. They understand our requirements and deliver expected results. Overall Sprint is very happy customer of Computaris TOP Tool, keep up the good work and we expect the continued support.”
Key tool in the DevOps, cloud and 5G technology roadmap
Looking ahead to future projects in the area of cloud and 5G, Computaris TOP Testing Suite is a key tool in the client’s technology roadmap. CI/CD support, Jenkins integration, J-unit plugin support is recommended and this sets a great roadmap for the DevOps model, while the 5G HTTP2 demo provided built the client’s confidence to pursue the Computaris automated testing tool even for 5G cloud solutions. Also, according to Sprint:
“We would like to continue partnering with Computaris for future DevOps components like centralized logging (Elasticsearch, Logstash, Kibana, ELK code name); centralized statistics (Graphite); centralized deployment and configuration, (Chef); automated testing and regression support.”