5 - (IOP 20) An Implementation Log for Tracking Key Variables in near Real Time: Development and Qualitative Evaluation in the Context of a Clinical Trial Conducted in Community Mental Health

Background: Implementation of evidence-based treatments (EBTs) is complex and not always reported in detail. This presents a critical barrier to replicable findings that are translatable in real world clinical practice settings. Aligning with ABCT’s mission of promoting rigorous research on cognitive behavioral treatments, the goals herein were to: (1) describe the development of an Implementation Log to systematically track a wide range of key implementation variables in near real time, and (2) gather qualitative feedback about the log from study personnel.

Method: These goals were executed in the context of a parent hybrid effectiveness-implementation cluster randomized controlled trial to evaluate a cognitive behavioral sleep treatment (Transdiagnostic Intervention for Sleep and Circadian Dysfunction) implemented via facilitation in community mental health centers. The Implementation Log was derived from implementation reporting guidelines, frameworks, and taxonomies; piloting; and collaboration with facilitators in the trial. After development, the log was used for 17 months by study personnel (i.e., facilitators; N = 6). Then, the facilitators were interviewed about their perceptions of the log’s acceptability, appropriateness, and feasibility. Interviews were deductively and inductively coded by two coders (Cohen’s kappa range = 0.72 to 0.94).

Results: Per the deductive analyses, almost all facilitators thought the log was appropriate (n = 5 out of 6 facilitators). However, most facilitators did not find the log to be acceptable (n = 4) or feasible (n = 5). Per the inductive analyses, overarching themes of “positive qualities/likes,” “barriers/dislikes,” and “suggestions for improvement” emerged from the data. To summarize, facilitators appreciated the log’s value to the field. However, they felt the log was time consuming and did not fit with the flow of facilitation work. Facilitators had three main suggestions: (1) automatize the format, (2) offer additional training and supervision for the log, and (3) improve flexibility to adjust the log’s variables. Additional subthemes from the inductive phase will be presented.

Conclusions: This report presents a novel Implementation Log for tracking a range of key implementation variables in near real time. Although modifications may improve end-user experience, the log has the potential to improve the reporting, planning, and analysis of efforts to implement EBTs in real world clinical practice settings. Limitations and future directions will be discussed.