Background: Studies published earlier in the chain of evidence often reported a larger treatment effect than subsequent ones, what has been called the Proteus effect. It is unclear whether such a phenomenon exists in diagnostic test accuracy studies.
Objectives: To determine the presence of exaggerated test accuracy measures in the first or second published studies compared with subsequent studies.
Methods: We retrieved data from all meta-analyses (MAs) of diagnostic test accuracy published in the Cochrane Database of Systematic Reviews between January 2003 and January 2020. One MA with the largest number of studies was selected from each systematic review. Within each MA, diagnostic test accuracy was compared across studies according to their order of publication. We estimated the prevalence of having the largest sensitivity, specificity, or diagnostic odds ratio in the first two published studies.
Results: A total of 2,095 diagnostic test accuracy studies were included in the analysis. Each MA included a median of 14 studies (interquartile range [IQR]: 8-28) and 3,238 patients (IQR: 1,030-8,814). The prevalence of having the highest accuracy in the first two studies was in 27 MAs (29%) for diagnostic odds ratio, 18 MAs (20%) for sensitivity, and 23 (25%) MAs for specificity (Table 1). There was no significant difference between studies with early extreme findings and those without early extreme findings on number of patients and disease prevalence (Table 2).
Conclusions: Diagnostic test studies published early in the chain of evidence often reported the highest diagnostic accuracy compared with subsequent studies. It is unclear what causes this phenomenon. Decision-makers should act with caution on early diagnostic evidence.
Patient, public and/or healthcare consumer involvement: None.