Effect estimates can be accurately calculated with data digitally extracted from interrupted time series graphs
Background:
Interrupted time series (ITS) studies are frequently used to examine the impact of population-level interventions. Systematic reviews with meta-analyses including ITS designs may inform public health and policy decision-making. Reanalysis of ITS may be required for inclusion in meta-analysis. Although publications of ITS rarely provide raw data for reanalysis, graphs are often included, from which time series data can be digitally extracted. However, the accuracy of effect estimates calculated from data digitally extracted from ITS graphs is currently unknown.
Objectives:
To assess the accuracy of effect estimates calculated from digitally extracted data from ITS graphs.
Methods:
Forty-three ITS with available datasets and time series graphs were included. Time series data from each graph were extracted independently by four researchers using digital data extraction software. Data extraction errors were analysed. Segmented linear regression models were fitted to the extracted and provided datasets, from which estimates of immediate level and slope change (and associated statistics) were calculated and compared across the datasets. These estimates were then standardised for comparison across the datasets. For a particular dataset, standardisation was achieved by dividing the level and slope change estimates by the range of the outcome of the provided time series data (i.e., the maximum observed value of the outcome minus the minimum observed value); this yielded standardised effect estimates that could range from 0 to 1.
Results:
Although there were some data extraction errors of time points, primarily due to complications in the original graphs, they did not translate into important differences in the standardised level and slope change effect estimates (and associated statistics) (Figures 1 and 2).
Conclusions:
Using digital data extraction to obtain data from ITS graphs should be considered in reviews including ITS. Including these studies in meta-analyses, even with slight inaccuracy, is likely to outweigh the loss of information from noninclusion.
Patient, public and/or healthcare consumer involvement:
There was no involvement with patient, public and/or healthcare consumers.