Randomization
Randomization assigns eligible units to conditions by chance so that groups are comparable before the intervention on average.
Download Markdown (.md)Soroush’s Product Wiki · https://wiki.soroush.uk/topic/randomization/
01What it means
Randomization is deliberate assignment by chance rather than user choice. In controlled experiments it supports causal comparisons by balancing pre-treatment factors in expectation, subject to correct implementation and sufficient sample size.[1][2]
- 01Eligible unitsUser, account, cluster, or other independent unit
- 02Chance mechanismGenerate and persist assignment
- 03Comparable treatmentKeep eligibility, exposure, logging aligned
- 04Outcome observationMeasure under assigned condition
- 05Causal comparisonEstimate by assigned groups; document deviations
Balance is expected across repeated assignments, not guaranteed perfectly in one finite sample; implementation and data quality still require checks.
02Why it matters
Without credible assignment, feature effects can be selection effects. Randomization makes a treatment-control comparison stronger causal evidence than a self-selected or before-and-after comparison.
03When to use it
Use randomization for feasible, ethical interventions when estimating impact. Define unit, eligibility, persistence, allocation, and interference risks before launch.
04How it works
- Choose eligible population and assignment unit.
- Generate and persist the planned assignments.
- Keep eligibility, exposure, and logging comparable.
- Check allocation and baseline data quality.
- Estimate effects using assigned groups and document deviations.
05A practical example
A team randomizes organizations, not individual members, to a collaboration home because members share a workspace. It checks assignment integrity before comparing activation.
06Where it breaks
Randomization cannot fix missing events, noncompliance, spillovers, or a poor outcome definition. Some changes cannot be randomized safely or fairly.
Common mistakes
- Assigning by a unit that allows spillover.
- Allowing eligibility to differ by variant.
- Assuming assignment removes the need for data-quality checks.
07Sources
- ICH E9 Statistical Principles for Clinical TrialsInternational Council for Harmonisation · International guidelineICH E9 Statistical Principles for Clinical Trials
- NIST/SEMATECH e-Handbook of Statistical MethodsNational Institute of Standards and Technology · Government statistical referenceNIST/SEMATECH e-Handbook of Statistical Methods