Comparison of diagnostic test accuracy estimates using the Multiple Thresholds Model based on complete and published cut-off data with a bimodal reporting pattern
2Institute of Medical Biometry and Statistics, Faculty of Medicine and Medical Center – University of Freiburg, Freiburg, Germany, Germany
Background: The multiple thresholds (MT) model allows for diagnostic test accuracy (DTA) meta-analysis of multiple cut-offs simultaneously. Previous analyses demonstrated the ability of the MT model to approximate DTA estimates of a complete cut-off dataset, based on reported cut-offs. This has been done for the Patient Health Questionnaire-9 dataset (Benedetti 2020), which had a symmetrical cut-off reporting, and the most frequently reported cut-off were mostly the study-specific optimal cut-off. The optimal cut-offs defined by the authors are only a subset of the reported cut-offs, so most frequently reported cut-off is not necessarily the most frequent optimal cut-off. It has been speculated that the cut-off reporting pattern might influence the extent to which this approximation is possible.
Objectives: To evaluate how well MT approximates DTA of a complete cut-off dataset based on cut-offs with bimodal reporting where the most frequently reported cut-off is not the most frequent optimal cut-off defined by authors using Youden’s index.
Methods: We used data from an ongoing review on DTA of the Hospital Anxiety and Depression Scale questionnaire, for which information about published and all cut-offs was available. We present cut-off reporting pattern and DTA estimates obtained through the MT model for both published and all cut-offs.
Results: We analyzed 251 cut-offs from 13 studies. 81 cut-offs were published (Figure 1), while 170 were provided by authors. Reporting on cut-off was bimodal, 8 and 11 were the most frequent cut-offs, reported in 10 and 8 studies, respectively. Only one optimal cut-off each was at cut-off 8 and 11. Using reported versus full dataset, absolute difference of MT DTA estimates were as follows (Figure 2): sensitivities were underestimated by use of reported cut-offs up to 1.6% with cut-offs ≤10 and overestimated by a maximum of 1.4% above cut-off 10. Specificities were overestimated by a maximum of 3% with cut-offs ≤14 and underestimated by a maximum of 0.03% above cut-off 14.
Conclusions: Our preliminary analysis suggests that MT model approximates accurately the complete cut-off dataset despite the use of cut-offs with a bimodal reporting pattern.
Patient, public and/or healthcare consumer involvement: none Additional resources: Figure 1, Figure 2