Background: There is a growing awareness about the problem posed by untrustworthy randomized controlled trials (RCTs) in systematic reviews. It appears that some RCTs have been partially or entirely fabricated. We call trials subject to serious research integrity issues “problematic studies”, and recent examples can be found in reviews of ivermectin for the treatment of COVID-19. The Cochrane policy Managing Potentially Problematic Studies states that studies should not be included until serious concerns about trustworthiness have been resolved. However, there is no consensus around how to identify problematic studies. The INSPECT-SR (INvestigating ProblEmatic Clinical Trials in Systematic Reviews) project is developing a tool for identifying problematic RCTs in systematic reviews by combining empirical evidence with consensus methodology to develop a draft tool, which will then be refined through user testing. To date, the project has had input from over 70 people with expertise and experience in research integrity.
Objectives: 1. To introduce the draft INSPECT-SR tool to workshop participants. 2. To provide training in the draft tool by way of application to real examples. 3. To gather feedback from participants which will be used to improve the tool.
Description: This workshop will begin with a short (10 min) introduction to the draft INSPECT-SR tool and the workshop activity. Participants will work in groups to apply the tool to studies in Cochrane Reviews. The tool includes signaling questions in four domains. Illustrative examples are Inspecting results in the paper: Are the results substantially divergent from others in the meta-analysis? and Inspecting conduct, governance and transparency: Is the recruitment of participants plausible within the stated time frame for the research? The activity will be divided into 15-minute segments, corresponding to each domain. In each, participants will work in small groups to apply the signaling questions from one domain with support from facilitators (10 min), followed by facilitator-led discussion amongst all participants (5 min). The final 20 minutes will be devoted to overall discussion of participants’ impression of the tool, including capture of user feedback via questionnaire. Consumer involvement The project has an expert advisory panel representing a variety of stakeholders, including two lay members.