External validation and recalibration of the psychosis metabolic risk calculator in young adults with chronic psychotic disorders in the Netherlands

Background

Individuals with psychotic disorders experience substantially elevated cardiometabolic morbidity and mortality, yet risk prediction tools are seldom validated beyond first-episode or early-intervention settings. The Psychosis Metabolic Risk Calculator (PsyMetRiC), developed in the United Kingdom, estimates 6-year risk of incident metabolic syndrome (MetS) in young people with psychosis. Its transportability to chronic-care populations with greater metabolic burden remained uncertain.

Objectives

This study externally validated the full and partial PsyMetRiC models in a Dutch cohort of young adults with psychotic disorders receiving long-term psychiatric care, recalibrated the models for the Dutch setting, and evaluated their potential clinical utility.

Methods

Data were obtained from the PHAMOUS registry, a routine monitoring programme covering mental, social, treatment-related, and physical health outcomes in specialised psychiatric services in the northern Netherlands. Eligible participants were aged 16-35 years, had a psychosis-spectrum diagnosis, had no MetS at baseline, and had at least one follow-up assessment within 1-6 years. MetS was defined using harmonised International Diabetes Federation/AHA-NHLBI criteria. The full PsyMetRiC model incorporated age, sex, ethnicity, body mass index, smoking status, metabolically active antipsychotic use, high-density lipoprotein cholesterol, and triglycerides; the partial model excluded biochemical predictors. Missing data were handled using multiple imputation across 10 completed datasets, with estimates pooled using Rubin’s rules. Performance was assessed through C-statistics, calibration plots, calibration intercepts and slopes, Nagelkerke R², Brier scores, sex-stratified exploratory analyses, and decision curve analysis. Logistic recalibration updated the intercept and calibration slope to generate PsyMetRiC-NL models.

Results

The analytic cohort comprised 1,364 participants and differed materially from the original UK development sample: participants were older, more often male and White, more likely to smoke, had higher body mass index and systolic blood pressure, lower high-density lipoprotein cholesterol, and longer follow-up. The cumulative incidence of MetS was 29.1%. In external validation, discrimination was moderate, with C-statistics of 0.693 for the full model and 0.667 for the partial model. Both original models systematically underpredicted MetS risk in PHAMOUS, with positive calibration intercepts and slopes below 1. Logistic recalibration corrected calibration-in-the-large and calibration slope by design, while discrimination remained unchanged. Brier scores improved from 0.218 to 0.186 for the full model and from 0.207 to 0.191 for the partial model. Sex-stratified analyses suggested broadly comparable performance in men and women, although estimates for women were less precise. Decision curve analysis indicated that recalibrated PsyMetRiC-NL models provided greater net benefit than treat-all or treat-none strategies across clinically plausible thresholds, particularly from approximately 0.17 to 0.35.

Conclusions

In young Dutch adults with chronic psychotic disorders, PsyMetRiC demonstrated moderate discrimination but required local recalibration to achieve acceptable calibration. The recalibrated PsyMetRiC-NL models may support stratified metabolic monitoring, shared decision-making, and earlier preventive intervention. However, registry design, variable follow-up duration, missing-data assumptions, and absence of prospective impact evaluation constrain clinical implementation. Further validation and implementation studies are required before routine deployment.

References

No references