Test automation plays a major part of any continuous integration or continuous delivery pipeline.

When applying an approach such as as TDD (Test Driven Development) or BDD (Behavior Driven Development), our testing begins at a very early stage of the development life-cycle

Layers of automation and coverage

Automated testing may be broken down into a number of layers, but usually this consists of Unit tests (Classes, interfaces etc) which are run at compilation time and Acceptance tests (UI, Service layer etc) which are run after deployment.

Unit tests focus on code coverage, while Acceptance tests focus on business requirements coverage. Tools such as SonarQube are available, to determine the code quality of source code. They can be integrated into the build pipeline, to prevent poor quality code and the lack of unit tests from making it into production.

Modern tools such as Cucumber do not require a mapping back to requirements, as the scenarios themselves are the requirements. This prevents the need to create mapping matrices and other historical methods of proving the coverage of automation.

Getting the formula right

Determining the right mix of Unit testing, Service, Integration, User Interface, end to end and other types of tests generally fits into the automation pyramid guideline

Each project may differ slightly, some being heavier on User Interfaces while others being heavier on the service layer.

Continuous Integration/Delivery

Automation runs at various stages of the build pipeline, with different intentions at each stage. The closer we make it to production, the more realistic our environments are expected to be.

There are a number of advances which can be made which are not specifically mentioned in this diagram, such as running acceptance tests on local development machines or running performance and security tests earlier on.

This can be used as a guide, to how automation generally fits into the build pipeline