Psilocybin-based interventions for depressive symptoms: robustness and variation across meta-analytic specifications

Background: Psilocybin-based interventions have shown preliminary promise for depressive symptoms [1]. However, the evidence base is still small and methodologically heterogeneous, and single pooled estimates may depend on analytic choices. For instance, choices about outcome type, rater source, follow-up timing, comparator eligibility, open-label inclusion and risk-of-bias exclusion can all affect the pooled estimate. This motivates a specification-curve reanalysis of the dataset.

Aim: This study examines whether the estimated effect of psilocybin-based interventions on depressive symptoms is stable in direction and size across a range of plausible meta-analytic specifications.

Methods: I reanalysed the open SYPRES/Metapsy PSILODEP dataset (version 25.1.2) with 15 studies, 801 participants and 220 effect-size rows [1,2]. The main analysis was restricted to continuous post-baseline depressive symptom outcomes. Baseline and pre-dose rows, binary response and remission outcomes and post-crossover rows were excluded. As a result, 117 effect-size rows from 14 studies were available for analysis. For each specification, one effect size per study was selected, so that multiple outcomes from the same study were not treated as independent. Random-effects meta-analyses were fitted using restricted maximum likelihood (REML). A specification-curve approach was then applied [3]. Specifications varied across six analytic axes. These were dataset and comparator eligibility, rater source, follow-up timing, endpoint versus change-score preference, exclusion of high risk-of-bias studies and exclusion of open-label studies. This produced 216 specifications. Negative values of Hedges’ g favour psilocybin-based interventions.

Results: All 216 specifications produced valid pooled estimates. The median number of studies per specification was 8 (range 4 to 14). For comparison, I approximated the SYPRES primary model. This gave g = -0.95 (95% CI -1.21 to -0.69; k = 12), close to the SYPRES headline g = -0.90. Across all 216 specifications, pooled estimates ranged from g = -1.23 to g = -0.49, with a median of -0.86 (IQR -0.98 to -0.74). Every specification favoured psilocybin-based interventions, and every 95% confidence interval excluded zero. Median tau-squared was 0.15 (IQR 0.04 to 0.26). The size of the effect varied with analytic choice. Effects were larger for clinician-rated outcomes than for self-report (median g = -1.00 vs -0.63). Including open-label studies produced larger effects than excluding them (-0.96 vs -0.76). At the latest follow-up, effects were smaller than at the primary endpoint (-0.80 vs -0.91). Excluding high risk-of-bias studies produced effects similar to including them (-0.90 vs -0.84).

Conclusion: In every one of 216 specifications, the pooled effect favoured psilocybin-based interventions, and every 95% confidence interval excluded zero (median g = -0.86). The direction of the effect was therefore stable across the analysed choices. The size of the effect was less stable, and varied most with rater source and the inclusion of open-label studies. This is a study-level reanalysis. Future work could further address expectancy effects, blinding and publication bias. The value of this analysis is that it shows where the estimate is stable and where it varies.

References

1. Singleton SP, Sevchik BL, Lahey A, Cuijpers P, Harrer M, Jones MT, Nayak SM, Strain EC, Vandekar SN, Dworkin RH, Scott JC, Satterthwaite TD. A living systematic review, meta-analysis and open-data resource of randomized controlled trials of psilocybin treatment for symptoms of depression. Nature Mental Health. 2026. doi:10.1038/s44220-026-00630-8.
2. Singleton P, Harrer M, Sevchik B. metapsy-project/data-depression-psiloctr: Version 25.1.2 [dataset]. Zenodo. 2025. doi:10.5281/zenodo.17657950.
3. Simonsohn U, Simmons JP, Nelson LD. Specification curve analysis. Nature Human Behaviour. 2020;4(11):1208-1214. doi:10.1038/s41562-020-0912-z.