Background: The generalizability and applicability of meta-analyses from randomized controlled trials (RCTs) is often discussed and is part of the GRADE assessment. Objective: Our aim was to quantify generalizability using three prominent evidence syntheses from the field of critical care and perioperative medicine as examples.
Methods: Using three Cochrane reviews of interventions, we extracted the eligibility criteria of the included studies for each review. We used the PRECIS-2 score to assess the degree of pragmatism of each study. The topics were perioperative beta-blockers, vasopressors for shock, and hypothermia after cardiopulmonary resuscitation. We then defined cohorts of consecutive patients from central European hospitals according to the topic of each of the three Cochrane reviews, which included 2,241, 1,189, and 6,734 cases, respectively. We matched these patient cohorts with the respective eligibility criteria of the studies from the Cochrane reviews and calculated the proportion of patients in the cohort who met the eligibility criteria of the studies in each review. We also examined the association between the proportions of eligible patients and the PRECIS-2 score.
Results: The Cochrane reviews on perioperative beta-blockers included 36 RCTs, vasopressors for shock included 25 RCTs, and hypothermia after cardiopulmonary resuscitation included 12 RCTs. The proportion of patients who met the eligibility criteria for each trial varied widely, averaging less than 10%. A higher PRECIS-2 score was associated with a higher proportion of eligibility.
Conclusions: The standardized assessment showed surprisingly low generalizability, which may be explained in part by the degree of pragmatism of the individual studies within the evidence syntheses.
Patient, public and/or healthcare consumer involvement: None.