A research report by Apexon

White paper

The Role of Testing & QA in Digital Engineering

A Research Report by Apexon
Download White Paper
software testing and qa services
Executive summary

Digital is the New Normal

And it’s putting increasing pressures on Test/QA teams.

New digital technologies are enabling enterprises to re-think business processes, modernize the way they engage with customers, increase operational efficiency, and create new products and services.

The opportunities are endless, but success in digital requires new ways of thinking, and equally important, new ways of executing in the software development lifecycle. This is especially true in software testing and QA, where delivery teams need to move faster and more flexibly – without sacrificing quality or customer satisfaction. This means integrating development and QA efforts, automating manual processes wherever possible, and being ready to meet the unknown Testing and QA requirements of the latest digital technologies like IoT and AI.

In late 2019, Apexon surveyed QA/Test professionals and executives in the US and UK to assess the impact that digital engineering is having on their efforts. We wanted to understand the key challenges they face as well as the strategies they are implementing to keep pace in the world of digital.

Some of the key takeaways from the survey include:
Testing QA Takeaway
DIGITAL ENGINEERING

Digital engineering continues to put increasing pressure on Test/QA teams to adapt to new innovations and use cases while also ensuring high levels of software quality and customer satisfaction.

QA and Testing Takeaways Digital Engineering
ACCELERATE TESTING

There are also abundant opportunities for Test/ QA teams to leverage AI to accelerate their test automation efforts, but to date, this has not been embraced by most.

Key QA and testing takeaways
BUSINESS OUTCOMES

Test/QA teams are not linking their efforts to business outcomes as closely as they could. This is especially true as pressure increases for faster cycle times and more significant business outcomes from digital initiatives.

QA Testing Takeaway
Agile Squads

Continuous Testing is another big area of opportunity for Test/QA teams to align their testing strategies with the faster pace of digital engineering. It offers a way to make testing processes smarter by delivering more timely user feedback throughout the development lifecycle.

Key Takeaways
IOT AND AI EXPERTISE

As IoT and AI initiatives continue to gain traction in the enterprise, many Test/QA organizations are struggling to keep up due to a lack of knowledge and expertise.

Our Questions and findings

What are your organization’s top three QA & Testing objectives?

Despite the increased time pressures created by digital, software quality and customer satisfaction remain the top priorities for Test/QA teams.

  • 1These results may be driven by the fact that organizations are more focused on customer- facing considerations that have immediate impacts – both pro and con.
  • 2Interestingly, only 30% of the respondents considered security a top priority – at a time when new digital technologies are creating a lot of security unknowns for CISOs – new opportunities for bad actors.
  • 3We think it’s also essential to look at some of the longer-term needs that are being created by digital, e.g., adopting a secure, cloud-based infrastructure that enhances the speed and flexibility of digital development efforts.
organization’s top three QA & Testing objectives
More information

AWS Development & Testing for Cloud-Based Applications READ WHITE PAPER

Key Challenges for QA in Mobile, Web and IoT VIEW SLIDE SHOW

management expect from the QA & Testing team

What does your company’s executive management team expect from the QA & Testing team?

Executive expectations are well aligned with Test/QA team priorities.

  • 1Early software detection, increased software quality, and end-user satisfaction are the clear priorities for executives.
  • 2Contribution to business growth and business outcomes, reduced application cycle times, and increased quality and awareness across disciplines were ranked much lower.
  • 3Alignment between Test/QA and the business is a positive thing, but how much of this is shaped by pre-existing views and limited expectations of the breadth of the Test/QA team’s role?
  • 4What if the Test/QA team could demonstrate their ability to have a more direct impact on product cycle time, operational efficiency, and business growth and outcomes? In the world of digital, the need to develop and test with business outcomes in mind is crucial.
More information

How App Strategies are Evolving READ WHITE PAPER

Digital Predator Or Prey? Survival Of The Fittest In The Digital Economy Go To Website

How is the increased adoption of DevOps changing the role of QA and Testing in your organization software development lifecycle?

The integration of development and Test/ QA is now the standard, but some organizations still have a ways to go.

  • 1Over 60% of respondents incorporate Testing into development at some level.
  • 2With all the attention on DevOps in the last 10+ years, it’s surprising to still see over 35% of respondents still conducting most or all of their software testing after development has been completed.
role of QA and Testing while adoption of devops
More information

DevOps for Digital Go to website

Apexon DevOps Services Watch the video

evolution of digital

With the evolution of digital, which of the following factors are becoming inevitable for your QA teams?

Long testing cycles, tighter testing windows, and increasing complexity continue to challenge Test/QA teams in the age of digital engineering.

  • 1Interconnected systems and the plethora of devices, operating systems, and form factors make the Test/QA professional’s job more challenging.
  • 2Putting best practices into action and not having the right processes in place are also problematic.
  • 3Despite the fact that cycle time and speed were not determined to be top priorities for Test/QA teams, time-related issues are still at the top of the list of their challenges.
  • 4As digital innovation presents new use cases for mobile and multi-channel applications, we’re likely to see a lack of proper tools and testing methods become of even greater concern.
More information

Accelerating Your Digital Initiatives with Quality Engineering READ WHITE PAPER

Key Challenges for QA in Mobile, Web and IoT VIEW SLIDE SHOW

Digital is evolving at accelerated speed. Moving forward we must ensure that testing is integrated into the development process, rather than after development has been completed.

What are some of the major QA challenges in your company?

qa challenges in the organizations

What challenges do you face when testing mobile and multi-channel

(Mobile, Wearable, Social, Traditional) applications?
testing mobile and multi-channel challenges
More information

Accelerating Your Digital Initiatives with Quality Engineering READ WHITE PAPER

Key Challenges for QA in Mobile, Web and IoT VIEW SLIDE SHOW

What (in your opinion) will have the greatest impact on how QA and Testing evolve over the next two to three years?

Despite not being mainstream yet, many still expect AI to have a big impact on Test/QA in the future.
  • 1Nearly a third of respondents believe the digital transformation and the API economy will have the most significant impact, but AI is not far behind.
  • 2Cloud and cybersecurity are ranked surprisingly low, but that may be a function of both being seen as mainstream already, negating their potential impact in the future.
AI impact on QA/Testing
More information

How Software Testing Is Evolving by Leveraging AI Go to blog

Intelligent Testing for Digital View fact sheet

33%
Respondents believe Digital transformation & API will have the most significant impact on QA and testing
testing strategies for testing products IoT

Which of the following best describes your organization’s position?

In relation to having specific test strategies for testing products in an (IoT) environment?
  • 1IoT, as much as any area, requires new approaches to Test/QA.
  • 2In many cases, teams are being asked to come up with ways to test functionality that’s never been attempted before.
  • 3Some organizations need help putting effective strategies (and infrastructure) in place.
More information

IoT Testing Services Go to website

How to Tell if Your IoT Strategy is Rock Solid Go to blog

What percentage of your total applications are currently hosted in the cloud?

Cloud is the infrastructure of choice, with AWS and Azure leading the way.
  • 1Cloud is an essential enabler for the speed and flexibility required in digital engineering.
  • 2Users have much flexibility to which model they use – private, public, on-premise, and hybrid.
Cloud as choice of infrastructure
best cloud service providers

Which cloud providers does your organization use?

More information

AWS Development & Testing for Cloud-Based Applications Read white paper

Optimize Cloud Environments to Achieve Agile and DevOps Nirvana Go to website

Cloud is considered essential in digital engineering due to its ability to enable speed and flexibility.

Is your organization planning for any artificial intelligence projects in the coming year?

Artificial Intelligence projects are well underway in many enterprises and are likely to grow in importance as use cases mature.
  • 1Banking, smart plant (mfg.) management and automation are the most common use cases cited.
  • 2Yet, almost half of those surveyed did not have plans for an AI project in 2020.
artificial intelligence projects
More information

Cloud-Native Cognitive Computing How to Get Ready for the Next Frontier in AI Go to blog

Artificial Intelligence Services Go to website

What challenges did you encounter, or expect to encounter when setting up an artificial intelligence project?

challenges setting up an artificial intelligence project

What challenges did you encounter, or expect to encounter when setting up an artificial intelligence project?

Success with artificial intelligence is hampered by a lack of knowledge and expertise in multiple areas.
  • 1Connecting AI projects with clear business use cases is still the biggest challenge for enterprises. This is entirely consistent with other recent industry analysis.
  • 2In addition, the lack of knowledge in both development and QA is also hampering enterprise deployments of the technology.
  • 3The challenge of integrating AI technologies with other enterprise applications is also a factor.
More information

Artificial Intelligence Services Go to website

Get Ready To Accelerate in AI Go to blog

Does your organization currently use artificial intelligence to optimize quality assurance?

AI can help enterprises teams optimize their QA efforts, but adoption is still limited.
  • 1Over half the respondents are not doing anything with AI to support QA.
  • 2There are tremendous opportunities for AI—especially in the areas of test automation and predictive analytics, but Test/QA teams need guidance on how to move their AI efforts forward.
ai to optimize quality assurance
More information

How Software Testing Is Evolving by Leveraging AI Go to blog

Intelligent Testing for Digital View fact sheet

54%
Organizations are planning or have already planned for AI projects in the coming year

What (in your opinion) are the biggest challenges in achieving the desired levels of test automation?

challenges in test automation

What (in your opinion) are the biggest challenges in achieving the desired levels of test automation?

Test Automation offers significant benefits, but realizing them is a challenge.
  • 1Reduced test cycle time, better test coverage, and better reuse of test cases are a few of the significant benefits cited by respondents.
  • 2But they also cite several obstacles that prevent them from taking advantage of those benefits, including frequent changes to application code and a lack of skilled and experienced resources.

What benefits has your organization realized through test automation?

benefits of test automation
More information

Top 5 Test Automation Challenges & How to Solve Them Go to webcast

How to Flip Your Test Automation Pyramid for Agile Go to website

What is your current approach to maintaining the health of your test cases?

healthcare use cases testing

What is your current approach to maintaining the health of your test cases?

There are a variety of approaches to maintaining the health of test use cases.
  • 1Continuous integration is key to keeping test cases aligned with the latest versions of digital applications.
  • 2Others are parameterizing the test environment for test execution.
More information

Using DevOps Pipeline to Generate Continuous Insights Read white paper

The Latest in Digital Transformation Testing Services Go to website

Are you familiar with Continuous Testing?

Continuous Testing is another opportunity that enterprises are struggling to take full advantage of.
  • 1While there is broad awareness and use of Continuous Testing and its many benefits, less than 20% of respondents are using it extensively.
  • 2Users cite a lack of QA automation, DevOps, and skills as roadblocks.
continuous testing

Which of the following best describes your organization’s position?

organizations stand on continuous testing

Are you familiar with Continuous Testing?

continuous testing expertise in organizations
More information

Why Continuous Testing is Vital in Agile & DevOps Environments Read white paper

Continuous Testing Services Go to website

What has been the major roadblock in evolving towards Continuous Testing?

roadblocks in evolving with continuous testing
More information

Why Continuous Testing is Vital in Agile & DevOps Environments Read white paper

Continuous Testing Services Go to website

Continuous testing is still unfamiliar territory to many organizations. Over one third of respondents believe this is due to a lack of effective devops.

What is the scope of DevOps adoption in your organization?

DevOps has become a core requirement for enterprises pursuing digital engineering strategies.
  • 1Nearly half of the companies we spoke with have deployed DevOps on at least 50% of their development projects.
  • 2As a result, they are deploying code much more frequently.
devops used in more than 50% development projects
More information

AWS Development & Testing for Cloud-Based Applications Read white paper

Optimize Cloud Environments to Achieve Agile and DevOps Nirvana Go to website

How often does your organization deploy code?

how often code is deployed

Respondent profile

code deploy experts’ profile
More information

DevOps for Digital Go to website

Apexon DevOps Services Watch the video

integrate quality into development
To summarise
Integrating quality Into the development process is key
Conclusion
Success in digital requires quality at the core

The ability to get innovative new digital services to market more quickly while assuring optimal performance, functionality, and quality is a fundamental requirement for almost every business today at some level.

But this creates real pressures on delivery teams in the way of demand for more features, faster release cycles, tighter budgets, a constant stream of new variations and devices, and the need to incorporate new analytics and intelligence.

This requires delivery organizations to integrate and automate quality into their development processes more deeply than ever. The ability of organizations to make this shift across their delivery organizations will be the difference between success and failure in their digital efforts.