Skip to content

Research at St Andrews

Statistical model specification and power: recommendations on the use of test-qualified pooling in analysis of experimental data

Research output: Contribution to journalArticle

Author(s)

Nick Colegrave, Graeme D. Ruxton

School/Research organisations

Abstract

Common approach to the analysis of experimental data across much of the biological sciences is test-qualified pooling. Here non-significant terms are dropped from a statistical model, effectively pooling the variation associated with each removed term with the error term used to test hypotheses (or estimate effect sizes). This pooling is only carried out if statistical testing on the basis of applying that data to a previous more complicated model provides motivation for this model simplification; hence the pooling is test-qualified. In pooling, the researcher increases the degrees of freedom of the error term with the aim of increasing statistical power to test their hypotheses of interest. Despite this approach being widely adopted and explicitly recommended by some of the most widely cited statistical textbooks aimed at biologists, here we argue that (except in highly specialized circumstances that we can identify) the hoped-for improvement in statistical power will be small or non-existent, and there is likely to be much reduced reliability of the statistical procedures through deviation of type I error rates from nominal levels. We thus call for greatly reduced use of test-qualified pooling across experimental biology, more careful justification of any use that continues, and a different philosophy for initial selection of statistical models in the light of this change in procedure.

Close

Details

Original languageEnglish
Article number20161850
JournalProceedings of the Royal Society B: Biological Sciences
Volume284
Issue number1851
Early online date22 Mar 2017
DOIs
Publication statusPublished - 29 Mar 2017

    Research areas

  • Experimental design, Model simplification, Pseudoreplication

Discover related content
Find related publications, people, projects and more using interactive charts.

View graph of relations

Related by author

  1. Birdsbesafe® collar cover reduces bird predation by domestic cats (Felis catus)

    Pemberton, C. & Ruxton, G. D., 8 Oct 2019, In : Journal of Zoology. Early View

    Research output: Contribution to journalArticle

  2. Secondary dispersal mechanisms of winged seeds: a review

    der Weduwen, D. & Ruxton, G. D., Oct 2019, In : Biological Reviews. 94, 5, p. 1830-1838 9 p.

    Research output: Contribution to journalReview article

  3. Deconstructing collective building in social insects: implications for ecological adaptation and evolution

    Invernizzi, E. & Ruxton, G. D., 8 Aug 2019, In : Insectes Sociaux. First Online, 12 p.

    Research output: Contribution to journalReview article

  4. The Hermans–Rasson test as a powerful alternative to the Rayleigh test for circular statistics in biology

    Landler, L., Ruxton, G. D. & Malkemper, E. P., 7 Aug 2019, In : BMC Ecology. 19, 8 p., 30.

    Research output: Contribution to journalArticle

  5. A theory for investment across defences triggered at different stages of a predator-prey encounter

    Wang, L., Ruxton, G. D., Cornell, S. J., Speed, M. P. & Broom, M., 21 Jul 2019, In : Journal of Theoretical Biology. 473, p. 9-19 11 p.

    Research output: Contribution to journalArticle

Related by journal

  1. Proceedings of the Royal Society of London Series B: Biological Sciences (Journal)

    Oscar Eduardo Gaggiotti (Member of editorial board)
    1 Jan 2013 → …

    Activity: Publication peer-review and editorial work typesEditor of research journal

  2. Proceedings of the Royal Society B: Biological Sciences (Journal)

    Kate Arnold (Reviewer)
    2012 → …

    Activity: Publication peer-review and editorial work typesPeer review of manuscripts

  3. Proceedings of the Royal Society of London Series B: Biological Sciences (Journal)

    Michael Gordon Ritchie (Editor)
    20102011

    Activity: Publication peer-review and editorial work typesEditor of research journal

  4. Proceedings of the Royal Society B: Biological Sciences (Journal)

    Susan Denise Healy (Member of editorial board)
    2009 → …

    Activity: Publication peer-review and editorial work typesEditor of research journal

  5. Proceedings of the Royal Society B: Biological Sciences (Journal)

    Iain McCombe Matthews (Editor)
    2008 → …

    Activity: Publication peer-review and editorial work typesPeer review of manuscripts

Related by journal

  1. Advances in research on the impacts of anti-submarine sonar on beaked whales

    Bernaldo De Quirós, Y., Fernandez, A., Baird, R. W., Brownell, R. L., Aguilar De Soto, N., Allen, D., Arbelo, M., Arregui, M., Costidis, A., Fahlman, A., Frantzis, A., Gulland, F. M. D., Iñíguez, M., Johnson, M., Komnenou, A., Koopman, H., Pabst, D. A., Roe, W. D., Sierra, E., Tejedor, M. & 1 others, Schorr, G., 30 Jan 2019, In : Proceedings of the Royal Society B: Biological Sciences. 286, 1895, 20182533.

    Research output: Contribution to journalReview article

  2. Linguistic laws in chimpanzee gestural communication

    Heesen, R., Hobaiter, C., Ferrer-i-Cancho, R. & Semple, S., 13 Feb 2019, In : Proceedings of the Royal Society B: Biological Sciences. 286, 1896

    Research output: Contribution to journalArticle

  3. No evidence that warmer temperatures are associated with selection for smaller body sizes

    Siepielski, A. M., Morrissey, M. B., Carlson, S. M., Francis, C. D., Kingsolver, J. G., Whitney, K. D. & Kruuk, L. E. B., 24 Jul 2019, In : Proceedings of the Royal Society B: Biological Sciences. 286, 1907, 10 p., 20191332.

    Research output: Contribution to journalArticle

  4. Northern bottlenose whales in a pristine environment respond strongly to close and distant navy sonar signals

    Wensveen, P. J., Isojunno, S., Hansen, R. R., Von Benda-beckmann, A. M., Kleivane, L., Van Ijsselmuide, S., Lam, F. A., Kvadsheim, P. H., Deruiter, S. L., Curé, C., Narazaki, T., Tyack, P. L. & Miller, P. J. O., 20 Mar 2019, In : Proceedings of the Royal Society B: Biological Sciences. 286, 1899, 10 p., 20182592.

    Research output: Contribution to journalArticle

ID: 249951372

Top