(PS2-33) An Examination of Agreement Between Diagnosis and Treatment Goals in a Community-based Effectiveness Study

Abstract Text: Studies consistently demonstrate that diagnostic agreement between clinicians and researchers is poor, with clients often underdiagnosed, over-diagnosed, or diagnosed with the wrong disorder (Jensen-Doss et al., 2014; Verhoeven et al., 2017; Adams et al., 2024). This discrepancy is
attributed to differences in the diagnostic process; researchers often use evidence-based assessment techniques, such as structured diagnostic interviews, while clinicians rely on their clinical judgement to diagnose, which can lead to a misdiagnosis caused by information-gathering biases (Jensen-Doss & Hawley, 2011; Bowes et al., 2020; Yager et al., 2021). These findings are concerning in that diagnostic errors may affect the development of treatment goals and quality of subsequent treatment. However, this concern is based on the assumption that chart diagnoses reflect the focus of a client’s treatment, an assumption that is untested.
To examine this possibility, we will expand on the work of Adams et al., 2024, which found poor diagnostic agreement between clinicians and independent evaluators (IE) in a sample of 103 youths who participated in the Community Study of Outcome Monitoring for Emotional Disorders in Teens (COMET) study. The youths in the current study were 14.3 years old (SD = 1.6, range = 12-18) on average, largely cisgender female (66%, n = 68), heterosexual (57.3%, n = 59), Hispanic/Latinx (53.4%, n = 55), and White (53.4%, n = 55). The youths’ medical record diagnoses, assessed by community clinicians, and research protocol diagnoses, diagnosed by IEs as part of the study, were separately collapsed into diagnostic clusters: anxiety, depressive, trauma and stressor, obsessive-compulsive, and behavioral.
For this study, we are in the process of coding treatment goals documented in the youths’ medical record (n = 184 for 103 youth) into the same five clusters as the diagnoses. A quantitative content analysis approach will be used to analyze treatment goals. All treatment goals will be double coded and, when discrepant, discussed to consensus by the coding team, which will include a principal investigator of the parent study. Cohen’s kappa (1960) will be used to characterize correspondence between the treatment goals and medical record diagnoses to examine whether diagnoses are an indicator of treatment focus. We will interpret kappa coefficients using McHugh’s (2012) standards: ranges of 0-0.20, 0.21-0.39, 0.40-0.59, 0.60-0.79, 0.80-0.90, and >0.90 will be interpreted as having no, minimal, weak, moderate, strong, and almost perfect agreement, respectively. Given lack of prior research in this domain, this project is exploratory and no specific hypotheses are proposed, but high agreement between chart diagnoses and treatment goals would suggest chart diagnoses are good indicators of treatment focus. Implications for the dissemination and implementation of evidence-based practices will be discussed.