An experimental study to understand user experience and perception bias occurred by fact-checking messages

Sungkyu Park, Jamie Yejean Park, Hyojin Chin, Jeong Han Kang, Meeyoung Cha

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Fact-checking has become the de facto solution for fighting fake news online. This research brings attention to the unexpected and diminished effect of fact-checking due to cognitive biases. We experimented (66,870 decisions) comparing the change in users' stance toward unproven claims before and after being presented with a hypothetical fact-checked condition. We found that, first, the claims tagged with the Lack of Evidence' label are recognized similarly as false information unlike other borderline labels, indicating the presence of uncertainty-aversion bias in response to insufficient information. Second, users who initially show disapproval toward a claim are less likely to correct their views later than those who initially approve of the same claim when opposite fact-checking labels are shown - an indication of disapproval bias. Finally, user interviews revealed that users are more likely to share claims with Divided Evidence than those with Lack of Evidence among borderline messages, reaffirming the presence of uncertainty-aversion bias. On average, we confirm that fact-checking helps users correct their views and reduces the circulation of falsehoods by leading them to abandon extreme views. Simultaneously, the presence of two biases reveals that fact-checking does not always elicit the desired user experience and that the outcome varies by the design of fact-checking messages and people's initial view. These new observations have direct implications for multiple stakeholders, including platforms, policy-makers, and online users.

Original languageEnglish
Title of host publicationThe Web Conference 2021 - Proceedings of the World Wide Web Conference, WWW 2021
PublisherAssociation for Computing Machinery, Inc
Pages2769-2780
Number of pages12
ISBN (Electronic)9781450383127
DOIs
Publication statusPublished - 2021 Apr 19
Event2021 World Wide Web Conference, WWW 2021 - Ljubljana, Slovenia
Duration: 2021 Apr 192021 Apr 23

Publication series

NameThe Web Conference 2021 - Proceedings of the World Wide Web Conference, WWW 2021

Conference

Conference2021 World Wide Web Conference, WWW 2021
Country/TerritorySlovenia
CityLjubljana
Period21/4/1921/4/23

Bibliographical note

Funding Information:
This work was supported by the Institute for Basic Science (IBS-R029-C2) and the Basic Science Research Program through the National Research Foundation funded by the Ministry of Science and ICT in Korea (No. NRF-2017R1E1A1A01076400).

Publisher Copyright:
© 2021 ACM.

All Science Journal Classification (ASJC) codes

  • Computer Networks and Communications
  • Software

Fingerprint

Dive into the research topics of 'An experimental study to understand user experience and perception bias occurred by fact-checking messages'. Together they form a unique fingerprint.

Cite this