Background: We previously conducted an exploration of the trustworthiness of a group of clinical trials evaluating the effectiveness of cognitive behavioural therapy (CBT) and exercise in spinal pain from a single author team. We identified multiple concerns in eight trials, judging their data and conclusions to be untrustworthy.
Objectives: To systematically explore the impact of these trials (“index trials”) on results, conclusions, and recommendations of systematic reviews and clinical practice guidelines (CPGs).
Methods: We conducted forward citation tracking using Google Scholar and the citation chaser tool and searched the Guidelines International Network (GIN) library, and the National Institute of Health and Care Excellence (NICE) archive to June 2022 to identify systematic reviews and CPGs. We explored how index trials impacted their findings. Where reviews presented meta-analyses, we extracted or conducted sensitivity analyses for the outcomes pain and disability to explore how the exclusion of the index trials affected effect estimates. We developed and applied an ‘Impact Index’ to categorise the extent to which index studies impacted the results of identified meta-analyses.
Results: We included 32 unique reviews and 10 CPGs. None directly raised concerns regarding the veracity of the data from the index trials. Across all meta-analyses (55 comparisons), removal of the index trials reduced effect sizes by a median 58% (IQR 40-74). Using the Impact Index, 85% of comparisons were classified as highly, 3% as moderately, and 11% as minimally impacted. Nine out 10 reviews conducting narrative synthesis drew positive conclusions regarding the intervention tested. Nine out of 10 CPGs made positive recommendations for the intervention(s) evaluated.
Conclusions: This cohort of trials, with substantial concerns regarding trustworthiness, has had substantial impacts on the results of systematic reviews and the recommendations of CPGs, contributing to overly positive conclusions regarding the efficacy of adding CBT to physical rehabilitation for spinal pain. Patient, public, and/or healthcare consumer involvement: None.