1 - (SYM 75) Generative Artificial Intelligence-large Language Models' Ability to Detect Moral Injury in Combat-experienced Veterans

Moral injury (MI), a unique psychological stressor, often co-occurs in the context of traumatic experiences. MI is witnessing, failing to prevent, or participating in acts that violate deeply held moral beliefs, and is associated with maladaptive cognitive and emotional reactions. MI may hinder therapeutic engagement as people with MI hesitate to disclose these experiences (Onnink et al., 2024) and believe they do not deserve to feel better, avoiding therapeutic engagement (Norman & Maguen, n.d.). While the therapeutic relationship is important in reducing MI (Borges et al., 2019), initial assessment of MI is critical. Recent investigations indicate that generative artificial intelligence-large language models (GAI-LLM) can effectively assess the presence PTSD and suicide in veterans; however, research assessing GAI-LLM's ability to detect MI is non-existent.

For this investigation, six vignettes of a man or woman veteran with depression, PTSD, or MI (no PTSD) were provided to ChatGPT-4o, Google Gemini, and Claude (Levi-Belz & Zerach, 2023). The vignettes were identical with the exception of gender specific character names and pronouns and symptoms. The GAI-LLMs were asked to identify whether the character had a mental disorder, the disorder itself, and how likely the character experienced MI. The GAI-LLM also completed a modified version of the Moral Injury Questionnaire-Military Version-Observer (MI-observer; Currier et al., 2015; Lauderdale & Daniel, 2019) to assess if the GAI-LLM identified MI. 

The GAI-LLM identified all vignette characters as having mental disorders; however, the MI vignettes were identified as having PTSD. No gender differences were found for any outcomes. Two ANOVAs were used to assess mean differences in likelihood of MI and MI-observer ratings. For both ANOVAs, the GAI-LLM by veteran group interactions were statistically significant. For likelihood of MI, the PTSD and MI veteran groups were rated higher than the depressed veteran group. Google Gemini rated the PTSD veterans as likely to have less MI than the other GAI-LLMs. For MI-observer ratings, the depressed and PTSD veterans had lower MI-observer ratings than the veteran experiencing MI. Google Gemini made the lowest MI-observer ratings.

Although the GAI-LLM did not reveal gender biases, it overidentified PTSD in MI vignettes. There was also substantial variation across the GAI-LLM in the identification of MI. These findings suggest that GAI-LLMs need further training to be used to effectively identify MI in veterans.