Understanding how people reason about aesthetic evaluations of artificial intelligence

Changhoon Oh, Seonghyeon Kim, Jinhan Choi, Jinsu Eun, Soomin Kim, Juho Kim, Joonhwan Lee, Bongwon Suh

Research output: Chapter in Book/Report/Conference proceedingConference contribution

3 Citations (Scopus)

Abstract

Artificial intelligence (AI) algorithms are making remarkable achievements even in creative fields such as aesthetics. However, whether those outside the machine learning (ML) community can sufficiently interpret or agree with their results, especially in such highly subjective domains, is being questioned. In this paper, we try to understand how different user communities reason about AI algorithm results in subjective domains. We designed AI Mirror, a research probe that tells users the algorithmically predicted aesthetic scores of photographs. We conducted a user study of the system with 18 participants from three different groups: AI/ML experts, domain experts (photographers), and general public members. They performed tasks consisting of taking photos and reasoning about AI Mirror's prediction algorithm with think-aloud sessions, surveys, and interviews. The results showed the following: (1) Users understood the AI using their own group-specific expertise; (2) Users employed various strategies to close the gap between their judgments and AI predictions overtime; (3) The difference between users' thoughts and AI pre-dictions was negatively related with users' perceptions of the AI's interpretability and reasonability. We also discuss design considerations for AI-infused systems in subjective domains.

Original languageEnglish
Title of host publicationDIS 2020 - Proceedings of the 2020 ACM Designing Interactive Systems Conference
PublisherAssociation for Computing Machinery, Inc
Pages1169-1181
Number of pages13
ISBN (Electronic)9781450369749
DOIs
Publication statusPublished - 2020 Jul 3
Event2020 ACM Conference on Designing Interactive Systems, DIS 2020 - Eindhoven, Netherlands
Duration: 2020 Jul 62020 Jul 10

Publication series

NameDIS 2020 - Proceedings of the 2020 ACM Designing Interactive Systems Conference

Conference

Conference2020 ACM Conference on Designing Interactive Systems, DIS 2020
Country/TerritoryNetherlands
CityEindhoven
Period20/7/620/7/10

Bibliographical note

Publisher Copyright:
© 2020 ACM.

All Science Journal Classification (ASJC) codes

  • Computer Networks and Communications
  • Human-Computer Interaction
  • Software

Fingerprint

Dive into the research topics of 'Understanding how people reason about aesthetic evaluations of artificial intelligence'. Together they form a unique fingerprint.

Cite this