Inclusion of Misanalyzed Stepped Wedge Trials in Meta-Analysis: Findings from a Simulation Study (Epidemiology, Biostatistics, and Public Health, 2025)
INTRODUCTIONThe use of the stepped wedge cluster randomized trial (SWT) design to assess the effect of interventions in real-world settings has gained considerable popularity in recent years [1]. In this design, all clusters begin in the control condition, and the intervention is sequentially and randomly rolled out until all clusters transition into the intervention condition [2]. The unique characteristics of this design–for example, the unidirectional crossover to the intervention, variation in the timing and duration of exposure across clusters, and repeated measurements within clusters–pose additional statistical challenges that must be addressed when defining models to evaluate the effects of interventions [3]. In the context of meta-analysis, the inclusion of outcome data from SWTs requires careful methodological consideration. While analytic approaches for incorporating data from parallel cluster randomized trials have been well-documented [4], these methods are frequently misapplied in practice, with many analyses failing to properly account for clustering [5]. Given the added complexity of SWT designs, the potential for analytical errors, such as model misspecification [6], is expected to be higher. Currently, no established methods exist for synthesizing evidence from SWTs within meta-analyses, underscoring an important methodological gap that warrants attention. OBJECTIVESThis study aims to examine the effects of including misanalyzed SWT in meta-analysis through a series of simulations. METHODSRCT and SWT datasets were simulated. RCT datasets were generated with a balanced two-arm design (treatment and control), assuming a normally distributed treatment effect with a mean difference of 0 and a fixed sample size of 1000 participants per study. SWT datasets were generated based on a repeated cross-sectional design with 5 time points, 50 observations per cluster, an average treatment effect of 0, an error variance (σ2) of 5, and a random cluster effect (τ2) of 1. Eighteen data-generating scenarios were considered, varying in number of clusters (20, 40 or 60), random treatment effect (η2 = 1, 2 or 3), and random time effect (absent (γ2 = 0) or present (γ2 = 1)). For simplicity, no correlation between random effects was included. Each SWT dataset was analyzed using three linear mixed-effects models:• Unadjusted for time: y ~ treatment + (1 | cluster)• Hussey and Hughes model: y ~ treatment + time + (1 | cluster)• Extended model accounting for random treatment and random time effects, where appropriate:y ~ treatment + time + (1 | cluster) + (0 + treatment | cluster) + (1 | clustertime) For each model, the estimated treatment effect and standard error were extracted. Meta-analytic datasets were then constructed, each consisting of 3 RCTs and 1 SWT. To enable direct comparison across the three models, each SWT dataset was included in three separate but matched meta-analytic datasets per scenario. For each scenario, random-effects meta-analysis was done using the Sidik-Jonkman estimator of between-study heterogeneity. Model performance was assessed by calculating the mean of the pooled effect sizes, along with bias, model-based standard error, coverage, and the percentage of statistically significant results out of 1000 pooled effect sizes at p