(PS8-93) Virtual patients, real growth: A chatbot for enhancing empathic communication

Abstract Text:

Background: Therapist’s empathy is considered one of the most important predictors of psychotherapy outcomes (Elliott et al., 2018). Role-play can effectively enhance trainees’ ability to provide empathic responses, especially in handling challenging clinical scenarios (Bennett-Levy et al., 2009). However, given the time constraints of supervision, trainees may not have adequate opportunity to learn via high-quality role-play. There have been attempts to supplement clinical training with role plays via computer simulations (Murray, 2014), though given the limitation of technology, results often lacked verisimilitude. Large language models (LLM), like ChatGPT, are capable of producing logical and fluid human-like conversations (Biever, 2023). Incorporating a chatbot powered by an LLM in role-play practices can help trainees improve empathic communication skills without requiring more supervision hours. The goal of this study was to develop a chatbot for role-play practice with individualized constructive feedback, and to understand its perceived effectiveness and usability.

Methods: Chatbot was developed using ChatGPT 4o. Several stages of “prompt engineering,” that is, designing a set of input instructions for AI models to generate desired, accurate, and context-appropriate responses, were undertaken. Resulting prompt consisted of instructions to take on the role of a clinical psychologist who handles challenging patients. Examples of therapeutic and non-therapeutic responses were provided to help the model differentiate between helpful and harmful responses. Prompt contained additional instruction to 1) simulate a difficult patient and 2) provide individualized feedback based on users’ responses.
Chatbot User Experience: Upon logging in, users have a choice of four virtual patient profiles. After the user selects one patient, the chatbot provides background information about the patient and begins the conversation with the patient’s statement, inviting a response from the user. The chatbot analyzes the effectiveness of the response with thorough feedback, gives an example of a better approach, explains in detail why the example might work better, and invites the user to either try again or continue the conversation. The conversation continues until the user decides to discontinue.
Usability study: We will recruit 15 doctoral students in clinical psychology who have started their clinical training. Consenting participants will answer questions related to their demographic backgrounds, theoretical orientations, previous experience with LLMs, and confidence in their clinical communication skills. Participants will then join a video-recorded online meeting with a research assistant. Participants will interact with the chatbot and make comments about their experience during the interaction (i.e., “Think-Aloud” technique). After 10 minutes of interaction with the chatbot, RA will conduct a focus interview with the participant to ask open-ended questions regarding the participant’s experience with the chatbot. Results will be available at the 2025 ABCT.