Background: Randomized controlled trials (RCTs) are the cornerstone of Cochrane recommendations/ Timely detection of integrity problems is therefore of imminent importance. While the integrity of RCTs has long been taken for granted, it has recently become apparent that 30% of RCTs might be fabricated. Concerns about potentially fabricated studies can be addressed with editors or publishers, who handled them according to guidelines of the committee on publication ethics (COPE). Here, I report my experience as a whistleblower.
Methods: Between ‘17 and February ‘23 I wrote to editors of 917 articles (497 (54%) RCTs) by 121 authors from six countries (11 between ‘17-‘19, 56 ‘20, 222 ’21, 599 ’22 and 29 in ‘23) in the field of obstetrics/gynaecology, urology, pediatrics and infectious disease (COVID-19). Papers had been published in 74 different journals (1 to 89 concerns per journal) by five different publishers. Concerns included implausible time-lines and effect-sizes, discrepancies between publication and trial-registration, data-copying from other articles, plagiarism, authors having published fabricated studies elsewhere and wrong statistics.
Results: As of March 6th, 212 investigations have been concluded, of which 89 (42%) resulted in retraction, 71 (33%) in an expression of concern while in 52 (25%) no evidence of wrongdoing was reported. Median time to first decision was 18 months. While the majority of editors respected confidentiality, some editors copied me in while writing to authors, suggested a letter-to-the-editor, named me in the retraction message or published my name in an editorial. There were no standards for investigation. Original data were shared with me in 53 cases, of which 45 showed signs of data-fabrication. In one case, the journal limited its investigation to the data-set provided by the senior author, while denying a problematic data-set provided by an author that stepped down after publication. In another case, the journal did not assess available original data themselves but relied on assessment by an expert designated by the author.
Conclusions: In post-publication assessment of problematic studies COPE procedures are ineffective, and a majority of editors do not know how to handle them. This forms an immediate health risk. No patient involvement.