Quality Engineering Needs an Agile Mindset

With two decades of popularity behind it, Agile has become the go-to methodology for any company that wants to not only move fast and break things, but also incorporate a certain way of workplace management. And, unless you have been living in a cave or on a remote island, it is extremely likely that the word (and its associated processes) has floated across your radar.  

In fact, agile project management – Agile is the shorthand version – has become so ubiquitous in the tech sector that it is hard to imagine a time when this type of workflow wasn’t used. The reasons for its blanket adoption are simple: an agile mindset focuses on the early delivery of business value, the continuous improvement of both the product and its processes, scope flexibility, ongoing team input and the desire to deliver well-tested products that meet a customer’s needs.  

To put it into lay terms, companies that integrate the principles and values of the Agile movement – as set out in the original Manifesto for Agile Software Development – are more likely to have excellent insights into how a project is progressing and, importantly, where any pain points will appear. In the fast-paced world of the connected society, that can often be the difference between success and failure. 

Taking that into account, we need to consider not only how adopting Agile can accelerate your time to market but also the impact on software quality itself.  

The Agile Frameworks 

While Agile has been part of the tech lexicon for around 20 years, it is generally accepted that there are four frameworks which conform to industry standards – Scaled Agile Framework (SAFe), Disciplined Agile (DA), Large-Scale Scrum (LeSS) and Scrum@Scale (S@S).There is some argument that a fifth framework was developed by Spotify, but that is often seen as an organic and people-driven take on successfully scaling Agile within an organization.  

And while there are literally millions of words written about on how Agile works and its benefits (a recent Forbes article gives a good overview of the principles, for instance), the fact that the methodology is an accepted part of digital transformation tells its own story.  

These four frameworks can be differentiated as follows: 

SAFe 

The Scaled Agile Framework® (SAFe®) is a set of organization and workflow patterns for implementing Agile practices at enterprise scale.  

SAFe® was formed around three primary bodies of knowledge: Agile software development, lean product development, and systems thinking. The framework promotes alignment, collaboration, and delivery across large numbers of Agile teams 

Disciplined Agile 

DA, previously referred to as Disciplined Agile Delivery (DAD), is a learning-oriented process decision framework for IT solution delivery.  

The framework provides a solid foundation from which to scale Agile solution delivery within enterprise-class organizations. DA utilizes Scrum and Kanban, along with transformation knowledge in areas like HR and finance, governance, DevOps, portfolio management and more. DA is often considered more flexible and easier to scale than other methods. 

Large-Scale Scrum  

LeSS is essentially regular scrum applied to large-scale development.  

This approach is based on the idea that scaling frameworks should be minimalistic – include less rules, roles, and artifacts – to drive success. However, both LeSS and SAFe® share some common patterns: Scrum at the team level, many teams sharing a backlog, collaborative planning across multiple teams, along with the general principles of pull and self-organization that any smaller Agile team may be familiar with. 

Scrum@Scale 

S@S is an extension of the Scrum framework. It is generally adopted by organizations that have already implemented Scrum successfully at the team level and are looking to spread it throughout the organization.  

The main goal is to align growing organizations around one common and shared set of goals. Coordination is managed through a Scrum of Scrums, which is comprised of Scrum Masters from each team, and a MetaScrum, made up of product owners. 

Agile and Quality Engineering 

There is a consensus that operational expectations of software Quality Engineering (QE) programs continue to accelerate. In response to these business optimization strategies, many organizations are adopting Agile to achieve quality at a speed that satisfies company and market demand.  

The caveat is that there is a defined need to balance the requirement for quality with the need for speed. It should come as no surprise that both are required for success.  

The problem is that QE practices around a software development program evolve organically and respond to the pressing needs of the day, week, or month. As a result, the quality practices that result from this organic workflow are inconsistent and out of alignment with agile methodology.    

Delivery teams often struggle to define a QE program in a scaled agile model. In some cases, management shifts too far left, believing that quality can be achieved by just asking developers to do it. In others, there is a siloed QE organization, working in a linear model and creating obstacles to the promise of agility.   

In recent years, development teams have realized the pitfalls of keeping testing on the extreme right of the Software Development Life cycle (SDLC) – as was typical in the waterfall model that preceded the wider adoption of agile methodologies. For many teams, the cost of waiting until the very end of the SDLC was punitive.    

As a result of this undoubted revelation, we now have approaches like ‘Shift Left’ and ‘Continuous Testing’ – both of which introduce testing from the start of the development cycle and produce completely tested and releasable code at the end of each sprint. An Agile methodology to QE is impossible without these approaches.  

Shift left, for instance, is directly related to the concept of “test early and often” – an ingrained practice which gained prominence around the same time that the aforementioned Manifesto for Agile Software Development was published. When you incorporate testing earlier in the timeline, then it becomes easier to spot flaws and bugs, thereby reducing the potential for a product to be delayed due to the need to fix defects late in the established lifecycle.  

Continuous testing, by contrast, also offers significant advantages to testing teams, not least of which is that the developer can get almost immediate feedback on the software. This approach became more prevalent as the Agile methodology increased in popularity, albeit that the goals of continuous testing are to make certain that the project can progress as planned through the delivery pipeline.  

That offers its own set of challenges, which include but are not limited to automation that fails when external systems fail or the infrastructure itself is not scalable enough to maintain a continued execution of the test suite. The latter challenge, for example, can create an inability to deliver QA (Quality Assurance) requirements in a compressed time period.  

Ultimately, software quality can be tested and measured in many ways. And there is no set structure for which tests need to be conducted and when they are complete.  

The circumstance depends on the specific importance and risks of the software task and the client’s objectives for the item. This is regardless of whether the group is taking a shot at legacy code, a greenfield project (one which lacks constraints imposed by prior work) or when assets are accessible for testing.   

Testing and Quality with an Agile Mindset 

Every organization is different and there is no “one size fits all” approach. When deciding what large-scale Agile development model will work best for your organization, you must analyze the current/future needs and the constraints of your specific case.  

No matter which scaled Agile approach you choose, the scale needs to be identified. A simple and small-scale Agile team option might be found with a LeSS approach, while an enterprise-scale company may opt for SAFe®. 

Whatever the chosen path, it is generally understood that testing is a process of executing a program with the intent of finding errors. However, that changes when adopting Agile and lean development. 

In this case, we must challenge some general testing assumptions. These can be categorized as follows: 

• Testing must be independent and separate from development.  
• Testing cannot start before coding is finished.  
• Testing follows the sequence of –
  
  • Test case design,  
  • Test case execution, 
  • Test case reporting (a test waterfall).  
• There must be a separate test department.  
• There must be a test manager.  
• Testing must be done at the end.  
• Testing must be “well planned.”  
• There must be a “testing strategy” and a “master test plan.”  
• 100% coverage is too expensive.  
• 100% test automation is too expensive.  
• Testing requires a sophisticated test-management tool.  
• Testing must be done by ‘testers.
  

Leaving these assumptions unchallenged can be very limiting to the goals of Agile. 

For instance, “Testing can only start after coding is finished” prevents delivery teams from accelerating cycle time. Instead, we have to ask: “Is there any way I could work differently so that testing can start before coding is finished?”  

In an Agile environment, QE should be involved from the start and continue through delivery.  

The involvement does not stop with testing and logging bugs - QEs should be involved in all project meetings during testing to understand requirement updates and changes. The role of QE becomes more dynamic with the advancement in automation and DevOps practices.  

Agile Testers continually add business value to Agile teams through proactive participation.  

They design and execute automated functional tests within the test automation framework and develop and maintain functional Regression tests, and identify and track software defects 

With that in mind, you should instead think about these testing recommendations to support your Agile initiatives:  

• Avoid using complex testing terminology  
• Keep testing classifications simple  
• Don’t separate development and testing  
• Testers and programmers work together  
• Educate and coach testing  
• Create a community of testing  
• Do not have separate test automation teams  
• Feature team as test automation team  
• All tests pass–stop and fix  
• Have zero tolerance on open defects  

We should also factor in what the definition of “done” is from an Agile perspective. The term is extremely subjective and completely dependent on context, which means that there is no one answer that can be applied to every scenario.  

To a programmer, “done” relates to when he/she has committed and unit tested the code, with the build remaining green. A tester will consider the code “done” when they have tested it. And a project manager will mark it as “done” after a successful demo or rollout to customers.  

That said there are a lot of things should be considered and evaluated for relevance in the context of attempting to decide if a user story has been tested ’enough.’ These include performance, reliability, localization, supportability, testability, security, documentation, compatibility … all of which are areas of importance to think about in terms of how the product will perform in the real world.  

Quality Engineering should be Agile  

We all know that testing is not what it used to be. In many cases, the barriers between testing and programming have to be demolished.  

Testing and programming are two sides of the same activity done by the same people, but both have a defined impact on not only the time to market but the functionality of the digital product itself. Taking an Agile approach means that the purpose of the tests changes from finding defects to preventing them by writing the specifications as tests and from checking the implementation to driving the design.  

This fundamentally different perspective leads to vital changes in the way people work and work together. And while there is a noticeable shift from traditional QA towards QE, the underlying principles of testing remain the same. A flawed product should never be released into the consumer market, and inadequate testing processes can impact both brand recognition and customer satisfaction.   

An Agile attitude to the culture of quality within an organization is a good place to start, but just integrating agility into workflows is rarely enough. Companies need to know that these changes in attitude can take time to implement, a tremendous amount of patience and, ultimately, full support across that organization. True transformation requires change on several levels, but the ones that commit to an investment in Agile in their testing processes will be successful in the long run.  

Apexon’s digital engineers have already solved the toughest virtual challenges that our customers have presented to us. To find out how we can alleviate the pain points and accelerate your digital journey with Agile, contact us today. Alternatively, fill out the form below and let us know what you need.