Results Speak For Themselves: 45 Percent Test Optimization Improvement

Quality or speed? In many regards, this is a trick question. These days, it is not enough to prioritize one or the other. Enterprises need to advance both, at the same time. Agile methodologies, the Shift Left mindset and DevOps techniques have all helped software professionals enormously in speeding cycle times and improving quality.

Still, there’s no doubt that Test and QA teams are feeling the pressure, as Apexon’s recent survey into the impact of digital transformation revealed. With more devices, operating systems and form factors than ever before, the task ahead might seem daunting for test and QA teams. Add to that the absolute need for seamless connectivity between devices, as well as the ability to incorporate user feedback quickly, and the scale of the testing challenge becomes clear.

Advancing automation with AI

Test and QA teams must by now be well aware of the critical importance of automation – or if you need a reminder, check out this blog on the subject. By automating a larger proportion of test cases, enterprises can effectively accelerate release time and come closer to achieving their digital transformation goals. While the adoption of automation is rising, it is still the case that the majority of enterprises have automated less than 25 percent of test cases.

For companies who are trying to increase test optimization with a view to increasing automation levels, the sheer quantity of test data to be analyzed can loom large. Applying techniques from the fields of artificial intelligence, machine learning and data analytics, enterprises can speed up test case optimization, improve error detection and focus their expert, manual resources where they really count.  To help with that, we recently launched ASTUTE, our own suite of AI-powered testing solutions, the only one designed for every stage of the test cycle.

AI-driven results analysis

Test result analysis is a great example of AI being put to work to deliver faster release times, while at the same time improving the quality of the end product. Why? Result analysis used to be a lengthy, error-prone process. Gleaning real quality improvements from the data was of course possible, but when the world around them moves at the speed of digital, enterprises can’t afford to lag behind with manual, cumbersome analytical processes. By the time the data is crunched, it is often a case of “too little, too late”.  AI-driven error categorization and analysis, on the other hand, provide enterprises with actionable intelligence, fast.

When we talk about AI-powered results analysis here at Apexon, we refer to a four-step process that can result in up to 45 percent improvement in test case optimization.

1. Bottleneck analysis: enables customized views for time, sprints, versions, components, projects, etc.

2. QA metrics drill-down: enables a deep dive from trends to actual root causes at the action step level.

3. Actionable intelligence: this is where teams can import and actually analyze test results, leading to an understanding of which features are impacting customer experience.

4. Error categorization: separates false negatives from real errors through analysis and error grouping.

The results are in…

Apexon recently worked with a client to analyze 150,000+ results with the goal of optimizing the process and identifying real application defects. (This comprised 25,000+ automated test cases, which were executed on four mobile devices and two browser configurations on a weekly basis.) By implementing configurable AI algorithms to identify defect patterns and automatically categorize them into error types, the client was able to allow its regression engineers to focus on actual failures with potentially critical consequences. It was easy and quick to spot the real errors. Algorithm-based defect bucketing enabled the client to focus resources where it matters.

Using Apexon’s AI-based analysis to look into execution time and infrastructure utilization, it even became possible to suggest optimization techniques for automation processes. All of these features combined meant that the client sped up releases cycles and improved software quality, while the solution was quick and easy to configure.

If complexity is slowing down your testing and QA, talk to us today about how to accelerate results and improve quality with Apexon’s Results Analyzer, part of the ASTUTE suite of AI-powered test solutions.

For a quick, easy way to check if our AI-driven solutions are right for your organization, complete this free assessment.