2026 ECNP Congress

EP01-1027

Leveraging large language models to extract social determinants from electronic health records to assess risk of acute psychiatric crisis

Sophie Riviere — N. Suarez, J. Colonel, S. Edwards, B. Todtfeld, M. Valenzano, T. Servedio, I. Melendez, A. Davidson, H. Thibeau, R. Jespersen, Y. Landa, P. Polosecki, B. Lin, S. Saha, J. Reinen, R.S. Kahn, C.M. Corcoran, G. Cecchi, S.S. Haas · Icahn School of Medicine at Mount Sinai, Psychiatry, New York, United States

2026 ECNP Congress

Introduction:

Social determinants of health (SDoH) play a critical role in shaping health trajectories and treatment access among psychiatric patients. Electronic health records (EHRs) contain rich patient-level data, including structured indicators of SDoH, and represent a potentially underutilized resource for examining SDoH on psychiatric outcomes. By focusing on a transdiagnostic cohort, we can identify social influences on health outcomes across all patients, rather than relying on diagnoses, which may evolve over the course of a patient’s outpatient treatment.

This study aims to examine associations between EHR-derived SDoH indicators and acute psychiatric crises. The secondary aim is to assess the accuracy of SDoH data extraction from intake notes and structured flowsheet data using a large language model (LLM).

Methods:

EHRs were derived from 625 patients (age: M[SD]=31.8[8.58]; 51.68% male) seeking initial outpatient psychiatric treatment. SDoH variables were derived from the SDoH Psych Standard flowsheet. Of the 23 domains, 12 items were retained (<25% missing data). Logistic regression analyses investigated associations between SDoH and the prevalence of acute psychiatric crises measured by either an emergency department visit or psychiatric hospitalization. Analyses were stratified by sex, and age was included as a covariate.

493 intake notes representing patients’ earliest documented engagement in mental health treatment were extracted from their EHR for SDoH analysis. An LLM prompt was developed to extract SDoH information from the clinical notes. To validate the LLM extraction pipeline, outputs from the LLM were compared with manual review and coding of a subset of 35 clinical notes by a human rater, with sensitivity and specificity calculated per SDoH variable in R using the confusionMatrix() function from the caret package.

Results:

In sex-stratified analyses, lack of insurance was significantly associated with acute psychiatric crises in both males (OR = 1.68, 95% CI [3.47, 8.30], p = 0.020) and females (OR = 1.65, 95% CI [3.38, 8.03], p = 0.023). Lack of employment showed strong positive associations with acute psychiatric crises in both males (OR = 1.58, 95% CI [4.04, 5.58], p < 0.001) and females (OR = 1.37, 95% CI [3.23, 4.77], p = 0.002). In males, lack of education was significantly associatedwith acute psychiatric crises (OR = 1.50, 95% CI [3.13, 6.41], p = 0.026), but not significant in females (ORs = 0.91, 95% CI [1.755,3.54], p =0.62).

The LLM generally showed good agreement with manual coding on average across all SDoH measures (mean sensitivity=0.98), but tended to over-extract SDoH categories, reflected in substantially lower average specificity (mean specificity=0.59), indicating a need for prompt refinement.

Conclusions:

Lack of employment status and insurance coverage are significant indicators of adverse psychiatric events at risk across sexes. Among males, lack of education is an additional indicator of risk. Exploratory analyses showed good correspondence between LLM-rater extraction of SDoH categories from clinical notes, though prompt refinement is needed to improve specificity. Critically, LLM-based extraction of SDOH data directly from clinical notes holds promise for filling gaps left by incomplete patient flowsheets and may ultimately reduce the burden of manual data collection by bypassing structured screening instruments altogether.

References

No references