Post hoc analysis
In the design and analysis of experiments, post hoc analysis (from Latin post hoc, "after this") consists of looking at the data—after the experiment has concluded—for patterns that were not specified a priori.
In practice, post hoc analyses are usually concerned with finding patterns and/or relationships between subgroups of sampled populations that would otherwise remain undetected and undiscovered were a scientific community to rely strictly upon a priori statistical methods.[citation needed] Post hoc tests, also known as a posteriori tests, are sometimes applied in exploratory research. They typically create a multiple testing problem. That is, each time a pattern in the data is considered, a statistical test is effectively performed. Multiple testing procedures are sometimes used to compensate, but that is often difficult or impossible to do precisely. Post hoc analysis that is conducted and interpreted without adequate consideration of this problem is sometimes called data dredging by critics because the more one looks the more likely something will be found.
See also
- ANOVA
- The significance level α (alpha) in statistical hypothesis testing
- Subgroup analysis
- Statistical power
- Testing hypotheses suggested by the data