What is reproducibility?
Independent verification of data and experiments is a cornerstone of scientific research. For this, it is important for experiments mentioned in research papers to be reproducible. In simple terms, reproducible research is achieved when the researcher cautiously conducts the entire research with complete attention to detail to ensure replicability at every step of the research process, regardless of the number of times the step or process is repeated.
Benefits of reproducible research
Replicability and accuracy are crucial elements that make scientific research rigorous, intensive, thorough, and trustworthy. They allow trust building among the scientific community and the lay public, facilitate efficient and quick analyses by other researchers and collaborators, and help in the progression of science by building new knowledge via faster reconfiguration of research experiments and tasks conducted previously.
The reproducibility crisis
Lack of transparency is an issue that impacts all disciplines. Especially in the fields of psychology and medicine, the reproducibility crisis is a nagging issue. Francis Collins (former NIH Director) states, “A growing chorus of concern, from scientists and laypeople, contends that the complex system for ensuring the reproducibility of biomedical research is failing and is in need of restructuring.” The results of the Reproducibility Project: Cancer Biology team’s studies (in eLife) showed that fewer than 50% of the experiments from high-impact research papers were replicable. A study by Nature found that more than 70% of researchers failed to replicate the results of at least one other scientist’s study, and more than 50% were unable to reproduce experiments from their own studies. In the US alone, public funding bodies spend around US$ 28,000,000,000 (US$ 28 billion) every year on studies that cannot be reproduced. Such observations and these huge costs contribute further to the lay public’s already dwindling trust in science.
Many who publish in open-access journals have a mandatory data availability statement (DAS) attached to their work. Authors claim to be willing to share data when journals ask them to submit data sharing statements along with their research papers. However, more than 90% of authors did not comply with their DAS, according to a recent study published in the Journal of Clinical Epidemiology. In most cases, no specific barriers were identified; the authors either did not respond or were unwilling to share data. Some challenges such as missing or lost data, lack of or failure to obtain ethical approval or informed consent for data sharing, authors no longer working with the same institution, etc. were also cited. This lack of data, resulting in the lack of reproducibility, wastes time, money, and effort; results in slower scientific progress; erodes public trust in science; diminishes scientific rigor and lowers efficiency; and adversely affects public health. “We have seen the problems caused by the lack of data sharing on COVID-19,” said Dr. Soumya Swaminathan, Chief Scientist, World Health Organization (WHO). “When data related to research activities are shared ethically, equitably, and efficiently, there are major gains for science and public health.”
Open data sharing – a possible solution to the reproducibility crisis
To this end, as of 25 January 2023, all National Institutes of Health (NIH) will mandate a data management and sharing plan for grant applications for projects that collect scientific data. This new NIH policy states that all the submitted data must be validated, the study findings should be replicable, and data should be shared using a credible repository. The policy requires that data sharing plans be made available preferably at the time of research inception. It is also necessary to include information on what types of tools and software are required to analyze the submitted data, the time when raw data were published, the location of raw data storage/publishing, and any specific access or distribution related considerations. The WHO has also mandated a similar data sharing policy this year (2022). This new policy states that all the health-related data from projects in which the WHO is involved need to be made accessible for reuse and sharing for the purpose of scientific research. The policy encourages authors to adhere to the FAIR Guiding Principles when sharing data, i.e., data must be findable, accessible, interoperable, and reusable.
These steps have been taken in order to make scientific, health-related, and biomedical data publicly available and mitigate the reproducibility crisis to an extent.
What do you think? Will these policies help solve the reproducibility crisis?
Open data sharing is a value add to scientific research, and CLS’ expertise in all things research can help you with all your publication needs. Together we can make a difference.