5 - (IOP 12) Implementation Outcomes from a Pilot Trial of a Redesigned Implementation Toolkit in Schools: A Mixed-methods Study

Background: Evidence-based practice (EBP) selection is often distributed across multiple educators and school leaders, and reactive to broader mandates or emergent crises. There is a need for multi-level decision-making tools that enable the systematic selection and adoption of EBPs in schools. Development of tools designed for community deployment using a context-specific approach is crucial to enhancing implementation outcomes. Our preliminary data suggest the relevance and potential of a multifaceted implementation toolkit redesigned for schools to support EBP selection and implementation. Further evaluation of this tool as part of standard educational practices is needed. We describe specific modification and redesign areas to a selection quality implementation toolkit for use in schools and present preliminary outcomes from the pilot test.

Methods: Using community-partnered, implementation science and human-centered design, we iteratively redesigned an existing toolkit to create the Adoption of Curricular supports Toolkit: Systematic Measurement of Appropriateness and Readiness for Translation in School (ACT SMARTS) for middle and high schools. ACT SMARTS includes implementation phase activities and tools, an implementation team, and an implementation facilitator who work together to complete toolkit activities. We collected quantitative data at multiple time points and completed a one-time qualitative interview from six participating school-based implementation teams regarding ACT SMARTS implementation outcome data (feasibility, acceptability, appropriateness, and usability).

Results: Participants rated Phases 1 (Exploration), 2 (Preparation), and 3 (Implementation) of ACT SMARTS as appropriate (M= 4.1, 4.2, 4.3, respectively) and acceptable (M= 4.0, 4.0, 4.3, respectively). Phases 1 and 2 were rated as moderately feasible, and Phase 3 as feasible (M= 3.7, 3.8, 4.1, respectively). Usability was in the moderate range (M= 59.8). Preliminary qualitative analysis provides further understanding of perspectives regarding implementation outcomes. Participants described activities in Phases 1 and 2 as usable and feasible, but noted challenges to engaging in Phase 3 of the toolkit with regard to the timing of this phase compared to the school calendar. Participants described meeting with toolkit facilitators as acceptable, appropriate, feasible and usable. Full in-depth qualitative analysis is underway.

Conclusions: Findings suggest that the redesigned ACT SMARTS is acceptable, appropriate, and moderately feasible and usable in middle and high schools. Areas for further refinement were identified to enhance its fit and impact in this new service context and prior to scale-up to improve the implementation of autism interventions in educational settings.