2 - (SYM 61) Can We Handle the Truth?
understanding Credibility and Engagement in Military Survey Responses

The military is often overlooked as a distinct culture with its own norms and practices necessitating representation and cultural competence in research endeavors (Bergman et al., 2014). Research involving military populations necessitates a comprehensive understanding of the unique military environment and requires careful consideration of the influence of cultural dynamics on research methodologies (Lincoln et al., 2016). For example, much of the research with military populations relies on self-report of psychological and health conditions that are associated with community stigma as well as very real career implications (Coleman et al., 2017). Prior studies in civilian populations have identified social desirability bias and disengagement as key challenges affecting the validity of self-report data (Meade & Craig, 2012) and survey credibility has long been the subject of empirical inquiry (Verba, 1971). In the current study we examined self-reported credibility in a large-scale survey of 3,338 active-duty military service members via quantitative and qualitative methods. Participants were subject to randomly dispersed attention check items as well as rated their “honesty” in responding to the survey (0-100 scale). Those scoring below 100 prompted to provide qualitative explanations for their lack of disclosure. Responses were coded to identify recurring themes, and attention-check items assessed overall engagement. For quantitative analyses, participants were categorized into four groups: 1) low credibility-disengaged, 2) low credibility-engaged, 3) credible-engaged, and 4) credible-disengaged. Results revealed the average endorsed level of “honesty” was 90.96 % with a standard deviation of 18.35 %. Interestingly, 16.1% of the sample respondents admitted a level of dishonesty at or below 80%, often citing confidentiality concerns, fear of consequences, and survey fatigue. Additionally, response patterns differed significantly across the four categories. These findings highlight critical gaps in self-report accuracy and underscore the need for improved methodologies to elicit more truthful and engaged responses. By identifying the factors contributing to less credible survey responses and disengagement, this study informs strategies to enhance survey design, ensuring more reliable data collection in military and other high-stakes environments. In doing so, it contributes to bridging the divide between research and practice, improving the utility of psychological assessments in informing policy.