Testing
Testing is the process of evaluating and verifying a software program to identify any bugs, defects, or deviations from its intended requirements. It involves executing the program under controlled conditions and analysing the results to ensure it
Purpose
The purpose of this testing is to ensure that a system behaves as expected under real-world conditions. Automated tests can capture a lot of issues but they also cannot validate everything such as user experience problems, unstable dependencies or edge-case scenarios.
Context
Industry Context
There is a direct correlation between the perceived quality and availability of software and business outcomes, as shown in the DORA research.
ZeroBlockers Context
The goal is for teams to work in a continuous deployment model, where they can release software after each commit. This means that there is no blocking testing phase. However, this does not mean that testing is not important, it just means that it happens in a different way. It is still crucial to have a good testing strategy in place to ensure that the software is of high quality and that it behaves as expected.
Methods
| Method | Description | Benefits |
|---|---|---|
| Dark Launching | Releasing features to production without user visibility to test performance and stability. | Validates backend changes under real conditions without affecting user experience. |
| Testing in Production | Deploying new features hidden behind flags, enabling testing in live environments without exposing them to all users. | Allows safe testing in production with a bonus that it enables staged rollouts and A/B testing. |
| Exploratory Testing | Unscripted testing focusing on discovering bugs by exploring software functionalities without predefined tests. | Encourages creativity and intuition that can uncover subtle issues. |
| Chaos Engineering | Introducing controlled disruptions to systems in production to test resilience and failure recovery processes. | Improves system robustness; identifies critical points of failure. |
Anti-patterns
- Launching to Production without Flags: Releasing features to production without hiding them or testing them in a controlled manner, risking user experience and system stability.
- Skipping Exploratory Testing: Relying solely on automated tests and missing out on the unique insights that human testers provide.
- Blocking Test Phase: Holding up the release process to conduct extensive testing, delaying the delivery of value to users.
Case Studies
Implementing Testing in Production for Legacy Systems
How Hudson's Bay Company successfully integrated testing in production to enhance the reliability and performance of their legacy systems.
Hudson's Bay Company
Enhancing Software Quality with Automated Testing
How Trimble leveraged automated testing to improve the quality and reliability of their software products.
Trimble
Enhancing Quality through Exploratory Testing
How organizations can improve product quality and team collaboration by focusing on exploratory testing