Chat but Verify: Combating Misinformation in Conversational Generative AI with Verification Affordance

Mengqi (Maggie) Liao & Sundar, S. S. (In press). “Chat but Verify: Combating Misinformation in Conversational Generative AI with Verification Affordance.” Journal of Computer-Mediated CommunicationAbstract: Does the conversationality in Generative AI tools like ChatGPT persuade individuals to readily trust the information they provide? If so, it can be a problem because these tools are known to hallucinate and even generate misinformation. Can the presence of a verification mechanism help mitigate this problem? Guided by the HAII TIME model, we addressed these questions with a pre-registered 2 (Gen-AI conversationality cues: low vs. high) × 4 (verification affordance: absence vs. cue vs. forced action vs. voluntary action) between-subjects online experiment measuring responses to a Gen-AI assistant that disseminates misinformation (N = 477). We found that high conversationality cues engender higher trust by inhibiting the activation of the negative machine heuristic (the perception that machines are mechanistic, incapable of subjective judgments, and cannot detect emotions, compared to humans). However, acting on the verification affordance mitigates this problem by decreasing the perceived credibility of the generated information. Our findings highlight the heuristic value of conversationality cues and reveal the underlying psychological mechanisms of lay users’ trust in generative AI applications that generate misinformation. Results underscore the importance of building effective verification affordances to reduce users’ overreliance on cognitive heuristics when making trust decisions. Theoretical and practical implications for human–AI interaction and the design of more socially responsible technology are discussed.