Embracing biological variation to improve reproducibility and reduce animal use

In the course of the so-called reproducibility crisis, various sources of poor reproducibility of animal research have been identified, including a lack of scientific rigor, low statistical power, analytical flexibility, and publication bias. However, the reproducibility of a finding depends also on its external validity. Unless a finding is robust against common sources of variation between independent replicate studies, it will not be reproducible. In animal research, effects of experimental treatments can vary depending on the animals’ phenotype, which can vary depending on the animals’ environment. Environmental differences between replicate studies can thus produce conflicting findings. As environmental differences between replicate studies are unavoidable, systematic variation (heterogenization) of the animals’ phenotype rather than rigorous standardization is needed to improve reproducibility. Heterogenization of study populations can be achieved in many ways, including variation in genotype (sex, strain), age, housing condition, or test procedure, by splitting experiments into several replicate batches or by multi-laboratory studies. Using a randomized complete (or incomplete) block design, we can analyze not only whether a treatment has an effect but also whether that effect is robust against environmental variation and thus more likely to be reproducible – without the need for a larger sample size. Incorporating biological variation into study design should thus be the default option for almost all study types of experimental animal research—including exploratory studies. Rigorous standardization to a single genotype, a single sex, and a specific environmental context can only be justified if either the outcome of interest is known to be robust against variation in these factors, or the research question is truly limited to that specific context. In all other cases, systematic heterogenization will be scientifically more valid, and when considering individual studies as parts of research programmes, will also minimize animal use und thus be ethically and economically preferable.

Learning Objectives:

1. Understand the relationship between external validity and reproducibility of results

2. Understand why standardization fails to guarantee reproducibility

3. Ability to explain how heterogenization can be achieved without a need for larger sample sizes