Exploring the concept of reproducibility and developing a needs-based tool for educating end-users about research reproducibility: A qualitative study
Ivan Buljan, Faculty of Humanities and Social Sciences in Split
2Department of Research in Biomedicine and Health, University of Split School of Medicine, Croatia
3Center for Evidence-Based Medicine and Healthcare, Catholic University of Croatia, Zagreb, Croatia
4Faculty of Humanities and Social Sciences in Split, Croatia
Background:The concept of research reproducibility has gained significant attention recently. However, studies assessing research reproducibility use the terms “replicability”, “repeatability”, or “reproducibility” interchangeably. This severely affects the definition, the scoring system, and the results of the studies focused on reproducibility, but also highlights the lack of research on the understanding of what “research reproducibility” is.
Objectives:We aim to explore the understanding of the concept of research reproducibility and its importance to stakeholders, gather information about tools that could help better understand reproducibility, and gather information for designing a website-based tool for educating stakeholders and end-users about research reproducibility.
Methods:In this qualitative study, we will conduct three focus groups with different stakeholders (researchers and teachers; funders, editors, and policymakers; and other stakeholders), each with a minimum of 8 participants. In the first part, we will explore their views of reproducibility and its perceived importance and understand which interventions would improve their understanding of reproducibility as a concept. In the second part of the focus group, we will present the participants with the initial version of the reproducibility network website, using their input to inform its development as a website-based tool for educating end-users about research reproducibility. We will use open-ended questions for the moderated discussion in focus groups. The focus group sessions will be recorded, and responses transcribed.
Results:We will analyze the results using an inductive approach by applying thematic analysis with no pre-defined themes. As this is an exploratory qualitative study, we expect that the outcome will be a better understanding of how stakeholders define and think about research reproducibility. The findings will serve as the baseline for a more extensive quantitative survey study and will be presented at the Cochrane Colloquium 2023.
Conclusions:Our qualitative study will provide a clear user definition of the concept of research reproducibility and will also serve as a consumer-aided test of our website for tools on research reproducibility.
Patient, public and/or healthcare consumer involvement:A further understanding of the concept of reproducibility will help with the expectations and evaluations in studies assessing reproducibility, thus positively affecting policy development processes.