Background: A key function of Cochrane South Africa (SA) includes building the capacity of stakeholders to conduct reviews, use review evidence to inform healthcare decisions, and raise awareness about the importance of evidence-based healthcare. In 2021, we launched a project to formally evaluate our training.
Objectives: To evaluate training delivered by Cochrane SA across four levels: reaction, learning, behavior, and impact.
Methods: We followed the Kirkpatrick model for evaluating training effectiveness, using a mixed-methods study design (Table 1). To assess reactions to the training, and changes in knowledge and behavior, participants completed online questionnaires at different timepoints: immediately after every event for reaction to training; immediately before and after selected events for learning; and for behavior change, participants completed an adapted version of the evidence-based practice questionnaire before and 3 months after selected events. To assess impact, or the use of knowledge in practice, we interviewed participants of selected events 6 and 12 months after the event. Quantitative data were analyzed using descriptive statistics, and qualitative data using thematic analysis.
Results: We hosted 18 training events including 1,027 participants from November 2021 to December 2022, including webinars, a protocol development course, and a primer in systematic reviews (Tables 2 and 3). Regarding reaction to training, participants’ feedback was overall positive (Figures 1 and 2). Only 223/1,027 participants completed evaluation forms; response rates from the monthly webinars were poorer compared with those from the smaller workshops. Changes in knowledge were assessed for two events, changes in behavior for one event, and impact for two events; the results for these will be presented at the Colloquium.
Conclusions: The Training Evaluation project is helping us refine and improve the planning, delivery, and impact of the training Cochrane SA delivers. Challenges include the poor response rate and the lack of dedicated resources for implementing the evaluation more efficiently. Knowing the impact of our training will help us understand what the return on investment is, as planning and delivering training is resource intensive.
Patient, public and/or healthcare consumer involvement: Our training may contribute to more evidence-informed decision-making in Africa. This would be beneficial for patients and other stakeholders.