In statistics, Duncan's new multiple range test (MRT) is a multiple comparison procedure developed by David B. Duncan in 1955. Duncan's MRT belongs to the general class of multiple comparison procedures that use the studentized range statistic q_r to compare sets of means.

Duncan's new multiple range test (MRT)is a variant of the Student–Newman–Keuls method that uses increasing alpha levels to calculate the critical values in each step of the Newman–Keuls procedure. Duncan's MRT attempts to control family wise error rate (FWE) at α_ew = 1 − (1 − α_pc)^k−1 when comparing k, where k is the number of groups. This results in higher FWE than unmodified Newman–Keuls procedure which has FWE of α_ew = 1 − (1 − α_pc)^k/2.

David B. Duncan developed this test as a modification of the Student–Newman–Keuls method that would have greater power. Duncan's MRT is especially protective against false negative (Type II) error at the expense of having a greater risk of making false positive (Type I) errors. Duncan's test is commonly used in agronomy and other agricultural research.

Criticism

Duncan's test has been criticised as being too liberal by many statisticians including Henry Scheffé, and John W. Tukey. Duncan argued that a more liberal procedure was appropriate because in real world practice the global null hypothesis H0= "All means are equal" is often false and thus traditional statisticians overprotect a probably false null hypothesis against type I errors. According to Duncan, one should adjust the protection levels for different p-mean comparisons according to the problem discussed. The example discussed by Duncan in his 1955 paper is of a comparison of many means (i.e. 100),when one is interested only in two-mean and three-mean comparisons, and general p-mean comparisons (deciding whether there is some difference between p-means) are of no special interest (if p is 15 or more for example). Duncan's multiple range test is very “liberal” in terms of Type I errors. The following example will illustrate why:

Let us assume one is truly interested, as Duncan suggested, only with the correct ranking of subsets of size 4 or below. Let us also assume that one performs the simple pairwise comparison with a protection level $\gamma _{2}=0.95$ . Given an overall set of 100 means, let us look at the null hypotheses of the test:

There are $100 \choose 2$ null hypotheses for the correct ranking of each 2 means. The significance level of each hypothesis is $1-0.95=0.05$

There are $100 \choose 3$ null hypotheses for the correct ranking of each 3 means. The significance level of each hypothesis is $1-(0.95)^{2}=0.097$

There are $100 \choose 4$ null hypotheses for the correct ranking of each 4 means. The significance level of each hypothesis is $1-(0.95)^{3}=0.143$

As we can see, the test has two main problems , regarding the type I errors:
1. Duncan’s tests is based on the Newman–Keuls procedure, which does not protect the familywise error rate (though protecting the per-comparison alpha level)
2. Duncan’s test intentionally raises the alpha levels (Type I error rate) in each step of the Newman–Keuls procedure (significance levels of $\alpha _{p}\geq \alpha$ ).

Therefore, it is advised not to use the procedure discussed.

Duncan later developed the Duncan–Waller test which is based on Bayesian principles. It uses the obtained value of F to estimate the prior probability of the null hypothesis being true.

Different approches to the problem

If one still wishes to address the problem of finding similar subsets of group means, other solutions are found in literature.

Tukey's range test is commonly used to compare pairs of means, this procedure controls the familywise error rate in the strong sense.

Another solution is to perform Student's t-test of all pairs of means, and then to use FDR Controlling procedure (to control the expected proportion of incorrectly rejected null hypotheses).

Other possible solutions, which do not include hypothesis testing, but result in a partition of subsets include Clustering & Hierarchical Clustering. These solutions differ from the approach presented in this method:

- By being distance/density based, and not distribution based.

-Needing a larger group of means, in order to produce significant results or working with the entire data set.

References

Duncan, D B.; Multiple range and multiple F tests. Biometrics 11:1–42, 1955.

Juliet Popper Shaffer; A semi-Bayesian study of Duncan's Bayesian multiple comparison procedure, Journal of Statistical Planning and Inference 82 (1999)

Donald A. Berry , Yosef Hochberg; Bayesian perspectives on multiple comparisons, Journal of Statistical Planning and Inference 82 (1999)

Rajender Parsad, Multiple comparison Procedures, I.A.S.R.I, Library Avenue, New Delhi 110012

Tables for the Use of Range and Studentized Range in Tests of Hypotheses H. Leon Harter, Champaigne, IL; N. Balakrishnan, McMaster University, Hamilton, Ontario, Canada; Hardback - Published Oct 27, 1997

External links

Critical values for Duncan's multiple range tests