Skip to content

Research at St Andrews

Statistical model specification and power: recommendations on the use of test-qualified pooling in analysis of experimental data

Research output: Research - peer-reviewArticle

DOI

Open Access permissions

Open

Author(s)

Nick Colegrave, Graeme D. Ruxton

School/Research organisations

Abstract

Common approach to the analysis of experimental data across much of the biological sciences is test-qualified pooling. Here non-significant terms are dropped from a statistical model, effectively pooling the variation associated with each removed term with the error term used to test hypotheses (or estimate effect sizes). This pooling is only carried out if statistical testing on the basis of applying that data to a previous more complicated model provides motivation for this model simplification; hence the pooling is test-qualified. In pooling, the researcher increases the degrees of freedom of the error term with the aim of increasing statistical power to test their hypotheses of interest. Despite this approach being widely adopted and explicitly recommended by some of the most widely cited statistical textbooks aimed at biologists, here we argue that (except in highly specialized circumstances that we can identify) the hoped-for improvement in statistical power will be small or non-existent, and there is likely to be much reduced reliability of the statistical procedures through deviation of type I error rates from nominal levels. We thus call for greatly reduced use of test-qualified pooling across experimental biology, more careful justification of any use that continues, and a different philosophy for initial selection of statistical models in the light of this change in procedure.

Close

Details

Original languageEnglish
Article number20161850
JournalProceedings of the Royal Society B: Biological Sciences
Volume284
Issue number1851
Early online date22 Mar 2017
DOIs
StatePublished - 29 Mar 2017

    Research areas

  • Experimental design, Model simplification, Pseudoreplication

Discover related content
Find related publications, people, projects and more using interactive charts.

View graph of relations

Related by author

  1. What is known and what is not yet known about deflection of the point of a predator's attack

    Humphreys, R. K. & Ruxton, G. D. 2 Mar 2018 In : Biological Journal of the Linnean Society. 123, 3, p. 483-495 13 p.

    Research output: Research - peer-reviewReview article

  2. A non-parametric maximum test for the Behrens–Fisher problem

    Welz, A., Ruxton, G. D. & Neuhäuser, M. Mar 2018 In : Journal of Statistical Computation and Simulation. 88, 7, p. 1336-1347 12 p.

    Research output: Research - peer-reviewArticle

  3. Intragroup and intragenomic conflict over chemical defense against predators

    Best, R., Ruxton, G. D. & Gardner, A. Mar 2018 In : Ecology and Evolution. 8, 6, p. 3322-3329 8 p.

    Research output: Research - peer-reviewArticle

  4. A review of thanatosis (death feigning) as an anti-predator behaviour

    Humphreys, R. K. & Ruxton, G. D. Feb 2018 In : Behavioral Ecology and Sociobiology. 72, 16 p., 22

    Research output: Research - peer-reviewReview article

Related by journal

  1. Quantifying uncertainty due to fission–fusion dynamics as a component of social complexity

    Ramos-Fernandez, G., King, A. J., Beehner, J. C., Bergman, T. J., Crofoot, M. C., Di Fiore, A., Lehmann, J., Schaffner, C. M., Snyder-Mackler, N., Zuberbühler, K., Aureli, F. & Boyer, D. 30 May 2018 In : Proceedings of the Royal Society B: Biological Sciences. 285, 1879, 20180532

    Research output: Research - peer-reviewArticle

  2. Sexual selection predicts species richness across the animal kingdom

    Janicke, T., Ritchie, M. G., Morrow, E. H. & Marie-Orleach, L. 16 May 2018 In : Proceedings of the Royal Society B: Biological Sciences. 285, 1878, 8 p., 20180173

    Research output: Research - peer-reviewArticle

  3. Flight range, fuel load and the impact of climate change on the journeys of migrant birds

    Howard, C., Stephens, P. A., Tobias, J. A., Sheard, C., Butchart, S. H. M. & Willis, S. G. 28 Feb 2018 In : Proceedings of the Royal Society B: Biological Sciences. 285, 1873, 9 p., 20172329

    Research output: Research - peer-reviewArticle

  4. High rates of vessel noise disrupt foraging in wild harbour porpoises (Phocoena phocoena)

    Wisniewska, D. M., Johnson, M., Teilmann, J., Siebert, U., Galatius, A., Dietz, R. & Madsen, P. T. 14 Feb 2018 In : Proceedings of the Royal Society B: Biological Sciences. 285, 1872, 20172314

    Research output: Research - peer-reviewArticle

  5. Imitation of novel conspecific and human speech sounds in the killer whale (Orcinus orca)

    Abramson, J. Z., Hernández-Lloreda, V., García, L., Colmenares, F., Aboitiz, F. & Call, J. 31 Jan 2018 In : Proceedings of the Royal Society B: Biological Sciences. 285, 1871

    Research output: Research - peer-reviewArticle

Related by journal

  1. Proceedings of the Royal Society of London Series B: Biological Sciences (Journal)

    Gaggiotti, O. E. (Member of editorial board)
    1 Jan 2013 → …

    Activity: Publication peer-review and editorial workEditor of research journal

  2. Proceedings of the Royal Society B: Biological Sciences (Journal)

    Kate Arnold (Reviewer)
    2012 → …

    Activity: Publication peer-review and editorial workPeer review of manuscripts

  3. Proceedings of the Royal Society of London Series B: Biological Sciences (Journal)

    Ritchie, M. G. (Editor)
    20102011

    Activity: Publication peer-review and editorial workEditor of research journal

  4. Proceedings of the Royal Society B: Biological Sciences (Journal)

    Healy, S. D. (Member of editorial board)
    2009 → …

    Activity: Publication peer-review and editorial workEditor of research journal

  5. Proceedings of the Royal Society B: Biological Sciences (Journal)

    Matthews, I. M. (Editor)
    2008 → …

    Activity: Publication peer-review and editorial workPeer review of manuscripts

ID: 249951372