Artificial intelligence (AI) algorithms are making remarkable achievements even in creative fields such as aesthetics. However, whether those outside the machine learning (ML) community can sufficiently interpret or agree with their results, especially in such highly subjective domains, is being questioned. In this paper, we try to understand how different user communities reason about AI algorithm results in subjective domains. We designed AI Mirror, a research probe that tells users the algorithmically predicted aesthetic scores of photographs. We conducted a user study of the system with 18 participants from three different groups: AI/ML experts, domain experts (photographers), and general public members. They performed tasks consisting of taking photos and reasoning about AI Mirror's prediction algorithm with think-aloud sessions, surveys, and interviews. The results showed the following: (1) Users understood the AI using their own group-specific expertise; (2) Users employed various strategies to close the gap between their judgments and AI predictions overtime; (3) The difference between users' thoughts and AI pre-dictions was negatively related with users' perceptions of the AI's interpretability and reasonability. We also discuss design considerations for AI-infused systems in subjective domains.
|Title of host publication||DIS 2020 - Proceedings of the 2020 ACM Designing Interactive Systems Conference|
|Publisher||Association for Computing Machinery, Inc|
|Number of pages||13|
|Publication status||Published - 2020 Jul 3|
|Event||2020 ACM Conference on Designing Interactive Systems, DIS 2020 - Eindhoven, Netherlands|
Duration: 2020 Jul 6 → 2020 Jul 10
|Name||DIS 2020 - Proceedings of the 2020 ACM Designing Interactive Systems Conference|
|Conference||2020 ACM Conference on Designing Interactive Systems, DIS 2020|
|Period||20/7/6 → 20/7/10|
Bibliographical notePublisher Copyright:
© 2020 ACM.
All Science Journal Classification (ASJC) codes
- Computer Networks and Communications
- Human-Computer Interaction