An unusual diagnostic test accuracy review: challenges in study identification, data extraction and quality assessment
Katrin Probyn, Cochrane Response
2Division of Epidemiology and Biostatistics, Department of Global Health, Faculty of Medicine and Health Sciences, Stellenbosch University, Cape Town, South Africa
3Department of Epidemiology and Data Science, Amsterdam University Medical Centers, University of Amsterdam, The Netherlands
4Department of Clinical Sciences, Liverpool School of Tropical Medicine, Liverpool, UK
5Division of Clinical Pharmacology, Department of Medicine, University of Cape Town, Cape Town, South Africa
6Department of HIV & Global Hepatitis Programme, World Health Organization, Geneva, Switzerland
7Desmond Tutu HIV Centre, University of Cape Town, Cape Town, South Africa
8Cochrane Response, Development Directorate, Cochrane Central Executive Team, Canada
9Professor emeritus, Department of Clinical Sciences, Liverpool School of Tropical Medicine, UK
Background:
This was a joint project of Cochrane Response and Cochrane Infectious Diseases Group. The aim of the review was to determine the accuracy of simple measures of antiretroviral therapy (ART) adherence for detecting non-suppressed viral load in people with HIV that could be utilized in resource-limited settings. Included studies measured adherence to ART in different ways and compared the level of adherence to viral non-suppression.
Objectives:
To describe the methodological challenges of conducting a diagnostic test accuracy (DTA) systematic review with studies that were not designed to evaluate DTA.
Methods:
The index tests were any measures of ART adherence that could be utilized in resource‐limited settings (self-report, tablet counts, pharmacy records or electronic monitoring). The reference standard was a non-suppressed viral load, as detected using nucleic acid testing technologies. We extracted the 2×2 data using pre-piloted extraction forms. We assessed the methodological quality using the Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) tool, which was tailored to fit the review question.
Results:
We included 51 studies. Most studies were not set out to assess the measure of adherence as a diagnostic test but used other study designs also reporting on antiretroviral adherence. By agreeing on a set of “rules” on how to apply QUADAS-2, we were able to achieve consistency between reviewers. We encountered high variability for all index tests, and the overall certainty of evidence was very low. No measure offered either a sufficiently high sensitivity or specificity to detect viral non‐suppression.
Conclusions:
Identifying studies, data extraction and quality assessment were challenging because the studies we included were not designed as diagnostic accuracy studies. However, we successfully managed to include these studies by tailoring the data extraction form and the QUADAS-2 signaling questions.
Patient, public and/or healthcare consumer involvement:
There was no direct patient involvement in this review. However, the results of this review are very relevant for people with HIV living in resource-limited settings where viral load testing is not universally available or not feasible. The interim results of this review informed the new World Health Organization (WHO) HIV guideline.