Manipulation check

Manipulation check is a term in experimental research in the social sciences which refers to certain kinds of secondary evaluations of an experiment. Manipulation checks get at the interpretation of participants of factors in an experimental setting.

Overview

In experiments, an experimenter manipulates some aspect of a process or task and randomly assigns subjects to different levels of the manipulation ("experimental conditions"). The experimenter then observes whether variation in the manipulated variables cause differences in the dependent variable. Manipulation checks are separate measured variables that show what the manipulated variables concurrently affect (besides the dependent variable of interest). The purpose of manipulation checks is to provide evidence that the construct (or concept) that the experimenter believes that the experiment manipulates are indeed the constructs actually manipulated. In other words, they provide a test of construct validity (Kerlinger 1974). Manipulations are NOT intended to verify that the manipulated factor caused variation in the dependent variable. Random assignment, manipulation before measurement of the dependent variable, and statistical tests of effect of the manipulated variable on the dependent variable verify this. Rather, manipulation checks provide evidence that the construct one believes was manipulated was what was manipulated. Thus, a failed manipulation check does not refute that the manipulation caused variation in the dependent variable, but rather indicates that the experimenter has not shown that variation in the manipulated variable creates differences in the construct (concept) that the experimenter believed that the manipulated variable represents. In contrast, when a manipulation creates signficant differences between experimental conditions in both (1) the measured manipulation check variable and the (2) dependent variable, the interpretation is that (1) the manipulated achieved significant variation in the construct thought to be the "cause", and (2) the manipulation (and the associated "causal" construct) explains variation in the dependent variable (the construct thought to be the "effect").

See also

• Experimental design