Comparison of diagnostic test accuracy estimates using the Multiple Thresholds Model based on complete and published cut-off data with a bimodal reporting pattern

Session Type
Screening and diagnostic test accuracy synthesis methods
Fomenko A1, Linde K1, Aktürk Z1, Schneider A1, Rücker G2, Hapfelmeier A1
1Institute of General Practice and Health Services Research, TUM School of Medicine, Technical University of Munich, Munich, Germany, Germany
2Institute of Medical Biometry and Statistics, Faculty of Medicine and Medical Center – University of Freiburg, Freiburg, Germany, Germany

Background: The multiple thresholds (MT) model allows for diagnostic test accuracy (DTA) meta-analysis of multiple cut-offs simultaneously. Previous analyses demonstrated the ability of the MT model to approximate DTA estimates of a complete cut-off dataset, based on reported cut-offs. This has been done for the Patient Health Questionnaire-9 dataset (Benedetti 2020), which had a symmetrical cut-off reporting, and the most frequently reported cut-off were mostly the study-specific optimal cut-off. The optimal cut-offs defined by the authors are only a subset of the reported cut-offs, so most frequently reported cut-off is not necessarily the most frequent optimal cut-off. It has been speculated that the cut-off reporting pattern might influence the extent to which this approximation is possible.
Objectives: To evaluate how well MT approximates DTA of a complete cut-off dataset based on cut-offs with bimodal reporting where the most frequently reported cut-off is not the most frequent optimal cut-off defined by authors using Youden’s index.
Methods: We used data from an ongoing review on DTA of the Hospital Anxiety and Depression Scale questionnaire, for which information about published and all cut-offs was available. We present cut-off reporting pattern and DTA estimates obtained through the MT model for both published and all cut-offs.
Results: We analyzed 251 cut-offs from 13 studies. 81 cut-offs were published (Figure 1), while 170 were provided by authors. Reporting on cut-off was bimodal, 8 and 11 were the most frequent cut-offs, reported in 10 and 8 studies, respectively. Only one optimal cut-off each was at cut-off 8 and 11. Using reported versus full dataset, absolute difference of MT DTA estimates were as follows (Figure 2): sensitivities were underestimated by use of reported cut-offs up to 1.6% with cut-offs ≤10 and overestimated by a maximum of 1.4% above cut-off 10. Specificities were overestimated by a maximum of 3% with cut-offs ≤14 and underestimated by a maximum of 0.03% above cut-off 14.
Conclusions: Our preliminary analysis suggests that MT model approximates accurately the complete cut-off dataset despite the use of cut-offs with a bimodal reporting pattern.
Patient, public and/or healthcare consumer involvement: none Additional resources: Figure 1, Figure 2