AI engineering glossary · Statistics inference
What is a/B test?
A/B test (also called Split test): A controlled online experiment that randomly splits users into two groups, shows each a different version, and compares an outcome metric to judge which version is better. It is the standard way to estimate the causal effect of a product change.
A/B test: an example
Half of 40,000 app users see a green Pay button and half see blue; you compare payment completion rates.
Related terms
More Statistics inference terms
- Adjusted R-squared
- Alternative hypothesis
- ANOVA
- Average treatment effect
- Bayesian inference
- Benjamini-Hochberg procedure
- Bias of an estimator
- Bonferroni correction
Where you learn it
A/B test is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.