Applying the Instrument to assess the Credibility of Effect Modification Analyses (ICEMAN) in a meta-analysis

Session Type
Workshop - training
Assessment of the certainty of evidence
Target audience
Investigators of aggregate or individual participant data meta-analyses
Level of difficulty

BACKGROUND: Subgroup analyses are relatively easy to perform but can be difficult to interpret. Credibility is often low, the risk of over-interpretation high. ICEMAN is a structured approach generated through expert consensus and provides 8 items for judging the credibility of subgroup effects identified in a meta-analysis. OBJECTIVES: Learn how to apply ICEMAN to an apparent subgroup effect and, if credible, how to design a corresponding GRADE summary of findings table. DESCRIPTION: Participants will apply ICEMAN to examples from the literature. We will discuss, for each item, conceptual and practical challenges. The workshop does not include statistical exercises; the focus is on the interpretation and reporting of subgroup effects. Participants should bring their laptop, if available, and are welcome to discuss their own examples. Key references: (1) Schandelmaier et al. CMAJ, 2020: Doi: 10.1503/cmaj.200077 (2) Guyatt et al. J Clin Epidemiol, 2023: Title: “GRADE Guidance 36: Updates to GRADE’s approach to addressing inconsistency” (in press)

Briel M1, Varadhan R2, Schmid CH3, Devasenapathy N4, Hayward RA5, Gagnier J5, Borenstein M6, Van der Heijden GJ7, Dahabreh IJ3, Sun X8, Sauerbrei W9, Walsh M10, Walsh M10, Ioannidis JP11, Thabane L10, Guyatt GH10
1University of Basel, Switzerland
2Johns Hopkins University, United States
3Brown University, United States
4The George institute for Global Heath, India
5University of Michigan, United States
6Biostat, United States
7University of Amsterdam, The Netherlands
8Sichuan University, China
9University of Freiburg, Germany
10McMaster University, Canada
11Stanford University, United States