AI engineering glossary · Statistics inference
What is permutation test?
Permutation test (also called Randomisation test): A hypothesis test that randomly shuffles group labels many times to build the distribution of a statistic under the null hypothesis of no group effect. The p-value is the share of shuffles at least as extreme as the real data.
Permutation test: an example
Shuffle 'A' and 'B' labels across 200 users 10,000 times; if only 120 shuffles show a gap as large as observed, p = 0.012.
Related terms
More Statistics inference terms
- Point estimate
- Population parameter
- Post-hoc test
- Posterior distribution
- Potential outcomes framework
- Power analysis
- Practical significance
- Prior distribution
Where you learn it
Permutation test is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.