Testing in Production
Testing in production refers to the process of testing new features, bug fixes, and updates in the live environment where the end-users operate. This approach complements pre-release testing by exposing new changes to real-world conditions.
Goal
The goal of testing in production is to ensure that new changes perform as expected under live conditions, ensuring that your testing effort is based on real-world scenarios and user interactions.
Context
Tests are traditionally performed in different environments such as development, staging, and pre-production. However, these environments may not fully replicate the real-world conditions, data and user interactions. In addition, we do not want to block the release process by having a long testing phase. Testing in production lets us achieve both aims.
Inputs
| Artifact | Description |
|---|---|
| Live Product | The code changes deployed to the production environment. |
| Feature Toggles | We use feature toggles to control the visibility of new features in the live environment to enable the testing of new features in production without exposing them to all users. |
Outputs
| Artifact | Description | Benefits |
|---|---|---|
| Defects Log | A compilation of defects, usability issues, and observations made during exploratory testing. | Provides immediate feedback and insights into application quality. |
Anti-patterns
- Skipping Pre-production Testing: Relying solely on testing in production without adequate pre-release validation can lead to critical issues affecting all users.
- Poor Monitoring Setup: Without comprehensive monitoring, identifying and diagnosing issues becomes challenging, potentially leading to prolonged user exposure to bugs.
- Exposing features to end users: Deploying new features to all users without proper testing can lead to a poor user experience and potential business impact.