Multiple Testing in Genetic Epidemiology

Abstract

If multiple statistical tests are performed simultaneously, we should take into account multiple testing to properly control the false positive rate. In association studies performing statistical tests for a large number of correlated markers, the traditional Sidák correction is overly conservative and the permutation test is inefficient. This article discusses recently proposed approaches for correcting for multiple testing in association studies. We first explain basic concepts of statistics such as the p‐value, false‐positive rate, corrected p‐value and family wise error rate. Then we discuss recently proposed methods in three categories: methods using multivariate normal distribution, methods calculating the effective number of tests and methods increasing the efficiency of permutation test. We compare the relative performance of these methods. Many of the methods are shown to be highly efficient and accurate compared to the traditional approaches and can readily be applied to the genome‐wide datasets.

Key Concepts:

  • In a statistical testing procedure performing multiple tests, multiple testing must be taken into account to properly control the false positive rate.

  • In association studies, the traditional Sidák correction is overly conservative and the permutation test is inefficient.

  • Recently proposed multiple testing correction methods are highly efficient and accurate and can be applied to the genome‐wide datasets.

Keywords: multiple testing; statistical test; p‐value correction; association study; false positive; family wise error rate; false discovery rate; permutation test; Bonferroni correction; multivariate normal distribution

Figure 1.

Probability density function of a bivariate MVN at two markers. The area outside the rectangle is the critical region, where the null hypothesis is rejected at any of the two markers. The outside‐rectangle probability is the corrected p‐value.

Figure 2.

Accuracy and efficiency of different multiple testing correction methods. We use the Wellcome Trust Case Control Consortium data. The vertical axis is the average error in corrected p‐values relative to the Bonferroni correction. The horizontal axis is the approximated time for correcting 10 genome‐wide p‐values for 500K SNPs assuming 100K permutations.

close

References

Altshuler D, Brooks LD, Chakravarti A et al. (2005) A haplotype map of the human genome. Nature 437(7063): 1299–1320.

Armitage P (1955) Tests for linear trends in proportions and frequencies. Biometrics 11(3): 375–386.

Benjamini Y and Hochberg Y (1995) Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal Statistical Society. Series B 57(1): 289–300.

Browning BL (2008) Presto: rapid calculation of order statistic distributions and multiple‐testing adjusted p‐values via permutation for one and two‐stage genetic association studies. BMC Bioinformatics 9: 309.

Cheverud JM (2001) A simple correction for multiple comparisons in interval mapping genome scans. Heredity 87: 52–58.

Conneely KN and Boehnke M (2007) So many correlated tests, so little time! Rapid adjustment of P values for multiple correlated tests. American Journal of Human Genetics 81(6): 1158–1168.

Dudbridge F and Gusnanto A (2008) Estimation of significance thresholds for genomewide association scans. Genetic Epidemiology 32(3): 227–234.

Dudoit S and van der Laan MJ (2008) Multiple Testing Procedures with Applications to Genomics. New York: Springer.

Gao X, Becker LC, Becker DM et al. (2009) Avoiding the high Bonferroni penalty in genome‐wide association studies. Genetic Epidemiology 34(1): 100–105.

Gao X, Starmer J and Martin ER (2008) A multiple testing correction method for genetic association studies using correlated single nucleotide polymorphisms. Genetic Epidemiology 32(4): 361–369.

Genz A (1992) Numerical computation of multivariate normal probabilities. Journal of Computational and Graphical Statistics 1: 141–150.

Han B, Kang HM and Eskin E (2009) Rapid and accurate multiple testing correction and power estimation for millions of correlated markers. PLoS Genetics 5(4): e1000456.

Kimmel G and Shamir R (2006) A fast method for computing high‐significance disease association in large population‐based studies. American Journal of Human Genetics 79: 481–492.

Leek JT and Storey JD (2008) A general framework for multiple testing dependence. Proceedings of the National Academy of Sciences of the USA 105(48): 18718–18723.

Li J and Ji L (2005) Adjusting multiple testing in multilocus analyses using the eigenvalues of a correlation matrix. Heredity 95: 221–227.

Lin DY (2005) An efficient Monte Carlo approach to assessing statistical significance in genomic studies. Bioinformatics 21: 781–787.

Moskvina V and Schmidt K (2008) On multiple‐testing correction in genome‐wide association studies. Genetic Epidemiology 32(8): 567–573.

Nyholt DR (2004) A simple correction for multiple testing for single‐nucleotide polymorphisms in linkage disequilibrium with each other. American Journal of Human Genetics 74: 765–769.

Nyholt DR (2005) Evaluation of Nyholt's procedure for multiple testing correction‐author's reply. Human Hereditary 60(1): 61–62.

Pe'er I, Yelensky R, Altshuler D et al. (2008) Estimation of the multiple testing burden for genomewide association studies of nearly all common variants. Genetic Epidemiology 32(4): 381–385.

Risch N and Merikangas K (1996) The future of genetic studies of complex human diseases. Science 273: 1516–1517.

Schaid DJ, Rowland CM, Tines DE et al. (2002) Score tests for association between traits and haplotypes when linkage phase is ambiguous. American Journal of Human Genetics 70: 425–434.

Seaman SR and MÃŒller‐Myhsok B (2005) Rapid simulation of P values for product methods and multiple‐testing adjustment in association studies. American Journal of Human Genetics 76: 399–408.

Sidák Z (1967) Rectangular confidence regions for the means of multivariate normal distributions. Journal of the American Statistical Association 62(318): 626–633.

Storey JD (2002) A direct approach to false discovery rates. Journal of the Royal Statistical Society. Series B 64(3): 479–498.

Storey JD and Tibshirani R (2003) Statistical significance for genomewide studies. Proceedings of the National Academy of Sciences of the USA 100(16): 9440–9445.

Wasserman LA (2003) All of Statistics: A Concise Course in Statistical Inference. New York: Springer.

Wellcome Trust Case Control Consortium (2007) Genome‐wide association study of 14 000 cases of seven common diseases and 3000 shared controls. Nature 447(7145): 661–678.

Westfall PH and Young SS (1993) Resampling‐based Multiple Testing. New York: Wiley.

Contact Editor close
Submit a note to the editor about this article by filling in the form below.

* Required Field

How to Cite close
Han, Buhm, and Eskin, Eleazar(Apr 2010) Multiple Testing in Genetic Epidemiology. In: eLS. John Wiley & Sons Ltd, Chichester. http://www.els.net [doi: 10.1002/9780470015902.a0022497]