Designing Experiments

Designing experiments is a systematic approach to testing hypotheses about how changes to a product or process might improve outcomes. It involves creating experiments to test these hypotheses under controlled conditions to evaluate the effects of various changes or features.


The purpose of designing experiments is to quickly validate, or invalidate, our assumptions before we invest too much time and effort into building a solution. It helps teams to make informed decisions based on data rather than assumptions.


Industry Context

We fall in love with our ideas so unless we have clear upfront clarification on the experiments we are going to run, the people that will be involved, their background/job-to-be-done and what represents success we will be biased towards our own ideas and create excuses for negative results.

ZeroBlockers Context

We are accountable for outcomes, not just outputs so we need to actively overcome our biases if we want to achieve the desired results.

ZeroBlockers Practices

Identifying Target AudienceTailor your evaluation methods to the audience.Increases the relevance and effectiveness of experiment findings.
Developing Experiment PlanOutline timeline, resources, and methodologies.organises experiment efforts and ensures efficient use of resources.
Recruiting ParticipantsEnsure a representative sample for your experiment.Enhances the validity and applicability of experiment findings.
Preparing Experiment MaterialsSuch as prototypes or alternatives for A/B tests.Facilitates effective and consistent data collection.


  • Confirmation Bias: Designing experiments to confirm pre-existing beliefs rather than to discover truth.
  • Ignoring Context: Failing to consider how external factors may influence the results.
  • Analysis Paralysis: Spending too much time planning and analysing, leading to delays in decision-making.

Case Studies

Was this page helpful?

Certainty-Importance Mapping
© ZeroBlockers, 2024. All rights reserved.