2026 ECNP Congress

PS03-3035

Facial emotion recognition using dynamic virtual avatars in schizophrenia and bipolar disorder: findings in immersive and non-immersive virtual reality

Mauro Scala — D. Sánchez-Reolid, R. Sánchez-Reolid, P. Fernández-Sotos, V. Romero-Ferreiro, M.Á. Alvarez-Mon, G. Lahera, A. Serretti, G. Fanelli, M. Colizzi, R. Arroyo Iturra, A. Fernández-Caballero, R. Rodriguez-Jimenez · Universidad Europea de Madrid, Faculty of Biomedical and Health Sciences, Madrid, Spain

2026 ECNP Congress

Introduction

Schizophrenia and bipolar disorder (BD) are associated with impairments in social cognition, particularly in facial emotion recognition (FER), which is crucial for interpreting emotions during social interactions [1]. Traditional FER assessments rely on static images or non-validated videos, limiting their ecological validity [2]. To address these limitations, we developed a set of dynamic virtual faces (DVFs) that use avatars to portray six basic emotions (happiness, sadness, anger, fear, disgust, and surprise) and a neutral expression [3]. Validation of this set of DVFs with 204 healthy controls (HCs) demonstrated that FER accuracy is comparable to traditional assessment tools, such as the “Penn Emotion Recognition Test” [4]. We performed two complementary studies to examine diagnosis- and sex-related influences on FER in these disorders.

Aims

  1. To compare FER accuracy between immersive and non-immersive virtual reality (VR) using 26 side-view DVFs in patients with schizophrenia relapse (n=10), BD manic episodes (n=10), and healthy controls (HCs) (n=10), and to provide a comparison of FER accuracy across these three groups.
  2. To explore sex differences in FER accuracy (i.e., the percentage of correctly identified facial expressions) in schizophrenia relapse (n=30) and healthy controls (HCs) (n=50). Participants completed a FER task including 52 DVFs (frontal and lateral views, high/low expressiveness) under immersive (VR headset) and non-immersive (monitor) conditions. All patients, hospitalized during an acute phase, were recruited between July 2023 and July 2024 at Hospital Universitario 12 de Octubre, Madrid, Spain. The software and DVFs are publicly available at the Castilla-La Mancha University repository RUIdeRA (https://hdl.handle.net/10578/27021).

Results

HCs (mean accuracy: 79.6% non-immersive VR; 81.9% immersive VR) performed significantly better than patients with schizophrenia (48.5%; 53.1%) and BD (53.1%; 53.9%) in both VR settings. Group differences in overall FER accuracy were significant for both non-immersive (p = 0.002) and immersive VR (p = 0.004). Post hoc tests revealed significant impairments in schizophrenia compared to HCs, with BD showing intermediate performance. No significant sex differences in overall FER accuracy were observed across VR conditions in schizophrenia (non-immersive p>0.05; immersive p>0.05) or HCs (non-immersive p=0.697; immersive p=0.100). Of note, female patients (66.1% in non-immersive; 68.8% in immersive VR) showed relatively preserved FER accuracy compared with healthy females (77.1% in non-immersive; 79.2% in immersive VR), with no significant differences between groups (non-immersive p=0.098; immersive p=0.156). In contrast, male patients showed pronounced deficits relative to healthy males across both VR settings (all p<0.001): non-immersive HC=80.1% vs schizophrenia=59.0%; immersive HC=87.4% vs schizophrenia=60.1%.

Conclusions

Comparison between side- and frontal-view DVFs revealed that overall FER accuracy was higher with frontal presentations, as previously reported by the same research group in an earlier study using only frontal-view stimuli [5]. Lateral perspectives reduce access to key facial regions and increase cognitive demands. Although no significant sex differences emerged, female patients with schizophrenia showed relatively preserved performance, whereas male patients exhibited marked deficits compared to healthy peers. These preliminary results highlight the potential of DVFs in VR as ecologically valid tools for assessing and rehabilitating social cognition, though larger and longitudinal studies are needed.

References

  1. Fernández-Sotos, P., Torio, I., Fernández-Caballero, A., Navarro, E., González, P., Dompablo, M., et al. 2019. Social cognition remediation interventions: A systematic mapping review. Plos One 14(6):e0218720.
  2. Collignon, O., Girard, S., Gosselin, F., Roy, S., Saint-Amour, D., Lassonde, M., et al. 2008. Audio-visual integration of emotion expression. Brain Res 1242:126–135.
  3. García, A.S., Fernández-Sotos, P., Vicente-Querol, M.A., Lahera, G., Rodriguez-Jimenez, R., Fernández-Caballero, A., 2020. Design of reliable virtual human facial expressions and validation by healthy people. Integr Comput-Aided Eng 27(3):287–299.
  4. Fernández-Sotos, P., García, A.S., Vicente-Querol, M.A., Lahera, G., Rodriguez-Jimenez, R., Fernández-Caballero, A., 2021. Validation of dynamic virtual faces for facial affect recognition. Plos One 16(1):e0246001.
  5. Scala, M., Sánchez-Reolid, D., Sánchez-Reolid, R., Fernández-Sotos, P., Romero-Ferreiro, V., Alvarez-Mon, M.Á., et al. 2024. Differences in emotion recognition between nonimmersive versus immersive virtual reality: preliminary findings in schizophrenia and bipolar disorder. Int Clin Psychopharmacol 40(5):281-287.