(PS5-13) Effectiveness of the audit and feedback implementation strategy to support the reach and fidelity of measurement-based care implementation in community mental health settings

Abstract Text: Measurement-based care (MBC) is a transdiagnostic and atheoretical evidence-based practice (EBP), which involves regularly administering questionnaires to clients and using the data to inform clinical decision making. Despite the evidence of effectiveness, and broad applicability, MBC is rarely implemented with high fidelity. Audit and feedback is a widely used and highly effective implementation strategy in medicine, which involves providing a summary of a target clinical behavior over a period of time to a clinician or clinical organization to highlight a gap between actual and ideal clinical practice. The few studies of audit and feedback applied to mental health EBPs showed it increased fidelity and was viewed favorably by stakeholders. This study used data from community mental health clinics implementing MBC (N = 4) to examine the effectiveness of audit and feedback in enhancing the reach and fidelity of MBC delivery. Reach was operationalized as the percent of eligible clients set up in the MBC system and the percent of eligible clients sent MBC measures. Fidelity was operationalized as the percent of MBC measures completed by clients and the percentage of completed MBC measures viewed by clinicians. Each clinic’s implementation for the last 30 days was audited from October 2021-April 2023 (N = 51 repeated measures per clinic per outcome). Reports began to be fed back to the clinics in April 2022. An interrupted time series design, using autoregressive integrated moving average (ARIMA) modeling, was used to examine changes in fidelity immediately and over time after the implementation of the audit and feedback strategy. To explore barriers and facilitators of implementing the audit and feedback strategy, qualitative interviews with leaders (N = 12) were analyzed using thematic analysis. The percentage of eligible clients set up in the MBC system had a negative slope before audit and feedback was implemented (β₁=-0.03, SE=0.12, p<.001), but rates were significantly boosted immediately after implementation (ω=0.38, SE=0.12, p=.002) and audit and feedback significantly positively changed the slope (δ=0.03, SE=0.01, p<.001). Audit and feedback did not significantly impact the percent of eligible clients sent measures. The percent of completed measures had a significantly negative slope before audit and feedback (β₁=-0.24, SE=0.12, p=.043), but was significantly boosted immediately after implementation (ω=3.62, SE=1.51, p=.018). Audit and feedback did not significantly impact the percent of measures viewed by clinicians. Qualitative results found leaders reported being motivated by clinical data, leaders varied in their perceptions of data accuracy, and clinics varied in how often and who presented feedback data in group supervision. Because audit and feedback is a low-cost and sustainable strategy, which eventually could be embedded within online MBC systems, it is important to understand the effectiveness of audit and feedback in supporting mental health EBP implementation. This study provides evidence that audit and feedback was effective in increasing some, but not all, indicators of reach and fidelity of MBC. Potential sources of variability were uncovered in qualitative analyses.