(PS5-38) Validation of the Implementation Climate Scale (ICS) in a Community-Based Learning Collaborative

Abstract Text: Despite the benefits of evidence-based practices (EBPs), implementation in community settings remains a challenge due to multilevel barriers at the individual, organizational, and systemic levels (Damschroder et al., 2009, 2022). Implementation climate, which assesses organizational support for EBP implementation (Ehrhart et al., 2014), is a key determinant of successful delivery of EBPs. Previous studies have validated the Implementation Climate Scale (ICS) across diverse settings (e.g., child welfare, substance abuse, nursing), with mixed findings on its factor structure, particularly concerning the Rewards for EBP subscale, which largely measures financial rewards not possible in all settings (Ehrhart et al., 2016, 2019; Peters et al., 2022; Lyon et al., 2018; Engell, 2020). This study aims to replicate and extend prior research by evaluating the psychometric properties of the ICS in a sample of 255 mental health clinicians, child welfare workers, and agency leaders participating in community-based learning collaboratives for Trauma-Focused Cognitive Behavioral Therapy and Trauma-Focused Case Management. This study is important because Trauma EBPs are not often implemented in low-resource settings, and this study further validates a measure of a key determinant of EBP implementation in a low-resource setting. Confirmatory factor analyses were conducted to assess the original six-factor structure, a five-factor model omitting the Rewards for EBPs subscale, and second-order latent factor model (i.e., a higher-level latent factor that helps explain covariation among first-level variables). While results indicated acceptable fit for both six-factor and five-factor models, the five-factor model, excluding the Rewards for EBPs subscale, provided a significantly better fit to the data (X2diff(40) = 76.78, p < .01), aligning with previous studies that found weaker support for this subscale in low-resource settings. A second-order latent factor model added to the five first-order factors (excluding Rewards subscale) also demonstrated improved fit when compared to the to the first-order five-factor model (X2diff(44) = 111.19, p < .01), supporting the use of an ICS total score. These findings validate the ICS for use in youth community mental health settings across provider types. However, results suggest that omitting the financial rewards subscale may improve applicability and interpretability, particularly in low-resource settings. Future research should further explore alternative incentive structures in different organizational settings, such as certification and credentialing, preferred caseload presentation with acquired EBP training, and organizational recognition. Additional research should examine if the rewards subscale is more appropriate in certain implementation contexts (e.g., state supported enhanced reimbursement).