What Is Agile Testing?

Introduction

In the world of software development, the term agile typically refers to any approach to project management that strives to unite teams around the principles of collaboration, flexibility, simplicity, transparency, and responsiveness to feedback throughout the entire process of developing a new program or product. And agile testing generally means the practice of testing software for bugs or performance issues within the context of an agile workflow.

Although the word agile is sometimes used to denote any kind of “dynamic” or “unstructured” way of working with others, the term more commonly suggests a focused yet rapidly iterative software process adhering to principles that were first outlined in a 68-word document called the gile Manifesto. Conceived by a group of software engineers in early 2001, the manifesto aimed to promote a more efficient, humane, and collaborative way of developing computer programs and IT systems. Today, the agile method—often referred to simply as Agile, with a capital “A” to denote its adherence to the principles outlined in the Agile Manifesto—has become widely accepted as an effective approach (or, more accurately, a class of approaches) to project management within the mainstream software development and testing community.

The Origins of an Agile Workflow

In February 2001 at a secluded ski lodge in Snowbird, Utah, a conclave of computer engineers conspired to envision the future of their industry. United by a shared passion for innovative, flexible approaches to project management and a distrust of more traditional linear methodologies, they sought to codify their zeal into a digital manifesto, a succinct statement of values that would be seen and pondered by developers and IT professionals worldwide. Through days and nights of intensive discussion, fueled by equal parts coffee and hot cocoa, seventeen men explored the state of the software industry, comparing notes on their favourite workflow practices—from “Scrum” to “Lean” to “XP” (i.e., Extreme Programming)—and working to find a synthesis of principles that would capture the best elements shared by these innovative paths.This synthesis, in turn, would serve to promote the dramatic changes they felt needed to be adopted and embraced by any software development team desiring to keep up with the ever-evolving realities of their field. Seeking more than to create another new project management method, they’d convened their wintry gathering to define a mindset, articulating the new order of thinking and working exemplified by dynamic approaches like Scrum and XP.

The Snowbird Accord that birthed the Agile Manifesto has taken on near-mythical proportions in certain software development circles. Handwritten notes from the historic meeting when the descriptive term “Agile” was agreed upon, along with printouts displaying the names and old email addresses of the chosen seventeen participants, are preserved and shared among developers and testers with a quasi-religious sense of awe. Within months of the gathering at Snowbird, a group of inspired converts gathered to form the Agile Alliance, an international non-profit organization dedicated to promulgating the pithy wisdom contained in the manifesto. To this day, the Alliance continues to host conferences and educational events, spreading the values of Agile worldwide, and training business leaders, IT professionals, and software developers to employ its methods successfully in their work.

Understanding the Agile Methodology

But in case you’re still wondering what, exactly, Agile is all about, its basic principles can probably best be explained in comparison to more traditional approaches to software development, such as “waterfall,” which Agile adherents believe to be an archaic holdover from an era when software first became a packaged commodity to be produced and sold to consumers like other physical goods. According to software testing expert Scott Barber, between 1979 and 1985 virtually all software companies began to codify their production processes into the same waterfall method that had been successfully used by the manufacturing and engineering industries for decades. In a waterfall approach, any given project’s workflow proceeds in a linear series of sequential steps, progressing down the chain of production from Requirements to Design to Implementation to Verification to Maintenance.

As you can see in the diagram above, in a waterfall process the “design” and “implementation” stages exist prior to, and distinct from, the “verification” and “maintenance” stages. This split between software developers and software testers, positioning them as separate entities at different points along a production cycle, is one of the fundamental problems that Agile seeks to resolve.In the Agile approach, developers and testers are seen as two sides of the same production coin, two parallel lines that should always meet and compare notes daily. From an Agile perspective, efficient production is severely hampered if your developers are striving to refine their code to a state of perfection before passing it on to a separate testing team, who then strive to break it in as many ways as they can before sending their damage report back to the dev team. This two-step process requires time, money, and frequently leads to internal division between a given company’s developers and testers. Instead, Agile suggests that these two essential functions be merged—not necessarily in terms of people, but in terms of time and process—thus bridging the illusory divide between code creators and code breakers, and even reducing the need for robust testing teams, while still respecting the necessity of both roles.

One could even say that in Agile, developers are encouraged to think more like testers, continually checking their own code for potential errors, and testers are encouraged to think more like developers, tempering their natural destructive tendencies to engage more fully in the creative process itself. Barber refers to this as creating a “product delivery team with a unified vision of delivering production-worthy code the ‘first’ time by integrating developer thinking and tester thinking throughout the code writing process.” This integration implies both that developers enhance their tester thinking skills and embrace the notion of direct interaction, even pairing, with those who specialize in tester thinking while coding.

Continuous Integration, Continuous Quality

In short, Agile requires a depth of continuous collaboration between stakeholders—including managers, developers, testers, and customers—throughout the production process that isn’t found in a traditional waterfall workflow. Testing becomes an essential component of each and every phase of the developmental process, with quality being “baked in” to a product at every stage of its development through constant feedback from everyone holding a vision of the final product. As Martin Fowler, one of the original signatories of the Agile Manifesto, 

explains: The key to this feedback is iterative development. This is not a new idea. Iterative development has been around for a while under many names: incremental, evolutionary, staged, spiral... lots of names. The key to iterative development is to frequently produce working versions of the final system that have a subset of the required features. These working systems are short on functionality, but should otherwise be faithful to the demands of the final system. They should be fully integrated and as carefully tested as a final delivery.

When testing in an Agile way, there are few, if any, rigid adherences to requirements documents and checklists. The goal, instead, is to simply do whatever’s necessary at any moment to satisfy a customer’s requests, replacing documentation with in-person meetings and replacing siloed functions with unified, self-organizing project teams. In an Agile company culture, everyone is expected to work closely together, no matter his or her role, to achieve a single goal: a high-quality software product that fulfils all of the essential specifications a client or designer requires with each iteration.Software developers, testers, and quality-assurance personnel wear each other’s hats from time to time, and while there may be a select group of people running most of the tests, the notion of a separate testing team disappears entirely for many product teams, and disappears within the core development cycle for those organizations who are required by external agencies (or even by law) to have formal and/or external release candidate testing. Even in these cases, this separate release candidate “testing” isn’t testing for the purpose of finding problems or improving the product, it’s an exercise in verification, regulatory compliance, and/or audit trail completion.It is this dynamically collaborative dimension to the Agile approach that its proponents believe sets it apart from other methods, with continuous internal dialogue leading to better communication with clients and end users, better responsiveness to ever-changing project requirements, better working relationships between developers and testers, and a better and more efficiently delivered software product overall.

While seemingly more complex than traditional project management methodologies such as waterfall, Agile’s proponents contend that, once successfully implemented, it quickly leads to greater simplicity, freeing up the creative potential and efficiency of software production teams in ways that sequential, linear, and more strictly procedural, by-the-book approaches to software development rarely do.Agile, as its name suggests, simply proposes to be a faster, more priority & risk focused, and more flexible, adaptable, and efficient way of conducting the complicated business of software production. And, of course, it is not without its critics.

Criticisms of Agile

“For all its proponents,” writes Mike Brown of uTest, “Agile has its fair share of skeptics and detractors. These are people who have a much different Agile experience—one characterized by chaotic processes, lower quality, miscommunication and numerous other problems.”And those critics are not in the minority. In 2012, Max Smolaks of TechWeek Europe,reporting on research conducted by Voke Media to determine what 200 different software companies thought of their attempt to embrace Agile, wrote:Out of over 200 participants, 64 percent said that switching to Agile Development was harder than it initially seemed. Forty percent of respondents did not identify an obvious benefit to the practice. Out of those who did, 14 percent thought it resulted in faster releases, and 13 percent—that it created more feedback. Seven percent of participants noted that Agile developers were happier due to reduced future planning and documentation.

These results strongly suggest that for organizations entrenched in years of non-Agile working practices, making the switch can be difficult, if not counterproductive. As SmartBear’s Scott Barber explains: I believe that the trend to “go Agile” is misguided. If a company is developing good software, the people involved in developing that software are happy working there, the software development is sustainable, and the business is being adequately served by that software, there’s really no need for them to try to be more or less Agile. Agile has challenges like any other culture, but the single biggest challenge I find is companies trying to solve development, process, management, and/or schedule problems by “going Agile.” Teams who have grown up in a culture that is fundamentally different than Agile simply will not find it easy to “go Agile.”In other words, companies expecting Agile to be a magic bullet to fix whatever ails their production efforts may be seriously disappointed, as it demands more of a systemic culture-shift than merely embracing a new set of tools and procedures. Moreover, it isn’t a one-size-fits-all approach. What works for a small San Francisco startup may be completely inappropriate for an enterprise with 5,000 employees distributed in cities around the world. Agile might not also be the right practice for software companies and IT firms that are subject to regulatory compliance mandates, such as government agencies and their contractors, where extensive documentation and procedures rife with checks and balances are essential.

Naturally, proponents of Agile suggest that virtually all of the problems, complaints, and negative experiences that companies attempting to use Agile have reported are due not to a problem with Agile itself, but with a failure to understand the approach and its limitations. Others, such as technology strategist Lajos Moczar, claim to understand Agile fully and yet contend that its guiding principles are flawed.In an article on CIO.com published in June 2013 and titled “Why Agile Isn’t Working,” Moczar ignited a firestorm, with pro-Agile and anti-Agile commenters on his post engaging in mild verbal warfare. “The comments in this thread have taken a strangely negative religious tone,” noted one participant, “like watching one sectarian argue doctrine with a member of another sect.” As with all ideological paradigms, some people embrace—or denounce—Agile with an almost spiritual fervor.However, there are many signs suggesting that despite its potential drawbacks, the rapid adoption and popularization of Agile is largely based on the pragmatic demands placed on software production and information technology in the early 21st century. When everything is changing so fast, some agility is simply required.

The Future of Software Development

As mentioned earlier, Agile has already won the battle for mainstream acceptance, and by all indications its general precepts are going to remain viable for some time. While much of this success is undoubtedly due to its effectiveness when used in appropriate circumstances, liberating software teams from old, tired ways of doing business, it’s also at least partially the result of the changing territory of computing itself.A couple of decades ago, when consumers would head to their local Costco to purchase software magnetically encoded on floppy disks, taking those over-sized boxes up to the register along with their bulk packages of breakfast muffins and corndogs, it made sense for major enterprises and small companies alike to view software as just another mass-produced industrial product. A waterfall-style production process seemed not only appropriate but also effective. Testers were necessary in order to vet developers’ code up until the final commit, whereupon quality-assurance product teams would oversee its transference onto physical media—with floppy disks rolling down the assembly line like any other physical product.

But then the Internet happened, and broadband changed everything. With the possibility of downloading software directly from publishers, along with periodic bug fixes and updates, the whole game changed. Production processes had to become iterative, lightweight, context-dependent, and more responsive than ever to customer feedback. To many, the possibility of ever “finishing” a software project seemed increasingly tenuous. The slow, step-by-step production-line mentality that had dominated software development in the 1980s was being challenged by the rapidly evolving infrastructure of computing itself. Something more flexible and efficient was needed to streamline the process, something more...agile. Today, we live in a world where digital downloads dominate sales and where continuous deployment is becoming the norm. At any given moment, your smartphone may be downloading background app updates to fix bugs you didn’t even know were there—pushed to your phone, from the cloud, by an Agile team that is used to refining new iterations of their software almost daily. The speed of software sales is only increasing (and increasing exponentially at that), and judging by reviews in the app stores, customers’ patience for software glitches seems to be decreasing conversely. For companies to remain competitive, running tests with the right tools as fast as developers can modify code, they have to realize that the future of software development is no longer linear—and perhaps never should have been.Until the next great innovation in software process comes along, the future is clearly Agile.