Background: JBI (formally known as the Joanna Briggs Institute) offers a suite of critical appraisal instruments that are freely available to evidence synthesisers. These instruments have been developed by JBI and collaborators and approved by the JBI Scientific Committee following extensive consultation. Following recent developments within the science of risk of bias assessment, it has been acknowledged that the existing suite of instruments are not aligned to these developments and conflate and confuse the process of critical appraisal with that of risk of bias assessment.
Objectives: Here, we introduce the revised critical appraisal tool for randomized controlled trials (RCTs) and detail the key changes made from the previous iteration.
Results:
Methods: The JBI Effectiveness Methodology Group (EMG) began the update procedure by cataloguing the questions asked in each JBI critical appraisal tool for study designs that employ quantitative data. These questions were ordered into constructs of validity (internal, statistical conclusion, comprehensiveness of reporting, external) following DELPHI-like methods. For questions that were related to the internal validity construct, they were further catalogued to a domain of bias through a series of mapping exercises. Finally, questions were then separated based on whether they were answered at the study, outcome, or result level. Findings: A strength of the JBI critical appraisal instruments has been their flexibility to facilitate assessments of risk of bias following different approaches. However, due to their presentation, using the tools following a domain-based approach was not intuitive to all users. By presenting the questions to the validity of construct they belong and the domain of bias they address, users of the tool are better placed to follow a domain-based approach using these new tools if they choose. The revised instrument also identifies which questions address internal validity and which questions should be addressed at different hierarchal levels.
Conclusions: The revision to the JBI critical appraisal instruments aims to provide greater flexibility to users of these tools. It is expected that this work will increase the usability and applicability of these instruments while maintaining consistency with modern advances in evidence synthesis.