Guidelines for Successful Cloud Test Automation in 2020

When an automation team spends a disproportionate amount of time on automating tests, and the resulting automation then ends up covering only about 30% of the tests, the automation policy has failed. A much higher percentage is needed to “test everything always” in Cloud applications. Additionally, automation should not dominate the test process. The actual cloud strategy and automation development and more importantly maintenance effort should only have a modest footprint in terms of time and resources.

While many testing organizations mistakenly approach automation from the perspective of tooling or programming, an approach centered on automation effective test design combined with an agile test development process yields far better results. When done right, the result is a set of automated tests with on-the-fly adaptability that readily meets the facile requirements of testing in the Cloud.

Tools have their place in the process, but frequently steal the center of attention, viewed as panaceas. The primary focus goes to buying and learning “the tool” rather than expending the time, effort, and cost involved in revisiting test design. But if a framework and test design process are not established, using a tool is like shooting in the dark — the Ready > Fire > Aim! approach adds up casualties quickly. The plan of attack must be mapped out before a proper weapon can be selected, otherwise “fire” can — and as we have seen, usually will — turn into “backfire”.

Establishing a test design process allows for more possible tests that are readily available, improving development cycles through flexibility. The approach aims to have at least 95 percent of tests automated, and 95 percent of testers’ time spent on test development, not automation.

These tests are not based on regression or bug validation but are calibrated to find and hunt for bugs, boundary conditions, state transitions, exploratory testing, and negative tests.

The test design approach has six essential principles:

1. No more than 5% of all tests should be executed manually cost of introducing automation is usually significant. By maximizing the investment, automating a high percentage of test cases, the return of a rewarding payoff is more likely.
2. No more than 5% of all efforts around testing should involve automating the testsCreating more and better test cases is key to proper test design. When testers spend significant portions of their time programming automation, test cases tend to be shallow, addressing only the basic functionalities of the system. Allocating time for in-depth development allows testers to write more elaborate cases, using testing techniques such as decision tables or soap opera testing, as well as their imagination (a frequently underestimated asset). The result is better coverage with less effort at the tool end.
3. Test development and automation must be fully separated to make sure that test cases are sufficiently in-depth, a distinction must be made between the responsibilities of testers and programmers. For successful Cloud test automation, testers must be dedicated only to testing.
4. Test cases must have a clear and differentiated scopeEach test case should be well-defined in its scope and purpose, and together test cases should map out comprehensive coverage while avoiding overlap and omissions.
5. Tests must be written in the right level of abstraction tools for conducting tests must be flexible enough to handle both higher business levels and lower user interface (UI) levels on demand.
6. Test methods must be simply the method used to achieve effective test design, and the subsequent high coverage automation should be easy and straightforward. Most of all, it should not add to the complexity of automation.

These principles of test design ensure the adaptability required for successful automated testing in the Cloud. Software development companies that have automated most of their testing, focus most of their efforts on testing, have dedicated testers trained to create comprehensive, flexible test design, and produce useful, accessible results will be well-equipped for travel at the speed of service.