(PS7-79) Small Effects, Large Impact: An Illustration from National Mental Health Data

Abstract Text:

Background: Even small changes in mean scores can have large population-level impacts, as recently argued by Carey et al., 2023. In this study, we provide an empirical demonstration of this using national data on paternal anxiety and depression symptoms from before and after the COVID-19 pandemic. We hypothesized that seemingly small effect sizes in mean score changes can translate to substantial increases in clinical symptom prevalence, demonstrating the need for contextualized interpretations.

Method: Data are from two nationally representative surveys conducted in 2018 (N=570) and 2024 (N=425) of fathers of children between the ages of 5 and 12. In both surveys, fathers completed the Generalized Anxiety Disorder-7 Assessment (GAD-7) and the Patient Health Questionnaire-8 (PHQ-8). Independent-sample t-tests were used to compare mean scores between the two time points, and Cohen’s d was calculated to determine the effect sizes of these differences. Additionally, we examined changes in the proportion of individuals scoring above the established clinical cutoffs for each measure to illustrate how even small mean shifts can translate into meaningful public health impacts.

Results: Mean anxiety scores increased from 2.97 in 2018 to 3.98 in 2024 (p <.001), with a small effect size (d=0.22). Mean depression scores also increased from 3.52 to 4.55 (p <.01) with a small effect size (d=0.20). Despite these small effect sizes, the percentage of fathers above each measure’s clinical threshold increased by 45.10% (from 10.2% to 14.8%) for anxiety and 55.36% (from 11.2% to 17.4%) for depression from 2018 to 2024.

Conclusions: These findings support the importance of considering the most appropriate measure of population-level effects. Even a slight change in mean scores at the population level can result in a much larger proportion of individuals exceeding clinical thresholds. Extrapolating these findings to the estimated 72 million fathers in the U.S. suggests that these mean increases could scale up to millions more fathers experiencing symptoms above the clinical threshold for anxiety and depression. Our findings support the argument that contextualizing effect sizes, such as by examining scores above clinical thresholds, is crucial for accurately assessing the real-world implications of research findings, particularly in public health contexts. This study underscores the need for nuanced interpretations of effect sizes and advocates for the adoption of supplementary measures to inform policy and intervention efforts.