(PS8-78) Is Seeing Believing? An Experimental Investigation of Users’ Ability to Identify Mental Health Misinformation on TikTok

TikTok has become a dominant source for media consumption, with over one billion active users. While this format creates opportunities for dissemination of information, it also facilitates the spread of misinformation. The proliferation of misinformation on TikTok is troubling given that users increasingly rely on it for mental health advice. Mental health misinformation can have detrimental consequences, such as steering users away from evidence-based care or promoting harmful treatments. With accurate and inaccurate information presented together on the platform, it is unclear whether users can distinguish between the two, and what factors influence their judgment. We experimentally investigated users’ ability to identify mental health misinformation in TikToks.

171 undergraduates viewed four TikTok videos about obsessive-compulsive disorder (OCD) created for this study. Information in videos was either accurate or inaccurate and presented by a speaker who was either a self-identified psychologist OCD expert or a person with lived experience. Each video was a combination of the two levels of the independent variables (Information: Accurate vs. Inaccurate and Source: Expert vs Testimonial). Participants were divided into two groups – High Knowledge and Low Knowledge – based on their performance on a pre-test of OCD knowledge. Videos were rated for accuracy and credibility on 5-pt scales from 1 (very inaccurate/uncredible) to 5 (very accurate/credible). We conducted a 2 x 2 RM-ANOVA with a between-subjects factor (Knowledge: High Knowledge vs. Low Knowledge) to test the effects of accuracy of information, source of information, and prior knowledge on perceived accuracy and credibility.

For accuracy ratings, there was a main effect of Source, p < .001, ηp2 = .24, with Expert videos (M = 4.22) rated as more accurate than Testimonial videos (M = 3.76), and a main effect for Information qualified by an Information by Knowledge interaction, p < .001, ηp2 = .09. Accuracy ratings for those with High Knowledge were significantly higher for Accurate (M = 4.21) than Inaccurate videos (M = 3.67), but not different for those with Low Knowledge (Accurate M = 4.08, Inaccurate M = 3.99). Credibility ratings resembled accuracy ratings, with a large main effect of Source, p < .001, ηp2 = .51 and a smaller main effect for Information that was qualified by an Information by Knowledge interaction, p = .009, ηp2 = .04, all in the same directions as for accuracy.

Results suggest that those without background knowledge were more susceptible to misinformation. Improvements in mental health literacy among the general public could decrease vulnerability to misinformation. Videos featuring speakers who identified as professionals were rated as more accurate and credible, regardless of the accuracy of the content they presented. This suggests that authoritative appeal is an effective rhetorical device for communicating mental health information. Experts could combat misinformation by making accurate videos that clearly state their credentials. A study limitation is a lack of external validity. Future research should continue investigating factors that inform perceived accuracy and credibility of mental health TikToks with more ecologically valid designs.