Messaging beyond texts with real-time image suggestions

Joongyum Kim, Taesik Gong, Kyungsik Han, Juho Kim, Jeonggil Ko, Sung Ju Lee

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

While people primarily communicate with text in mobile chat applications, they are increasingly using visual elements such as images, emojis, and memes. Using such visual elements could help users communicate clearly and make chatting experience enjoyable. However, finding and inserting contextually appropriate images during the chat can be both tedious and distracting. We introduce MilliCat, a real-time image suggestion system that recommends images that match the chat content within a mobile chat application (i.e., autocomplete with images). MilliCat combines natural language processing (e.g., keyword extraction, dependency parsing) and mobile computing (e.g., resource and energy-efficiency) techniques to autonomously make image suggestions when users might want to use images. Through multiple user studies, we investigated the effectiveness of our design choices, the frequency and motivation of image usage by the participants, and the impact of MilliCat on mobile chat experiences. Our results indicate that MilliCat's real-time image suggestion enables users to quickly and conveniently select and display images on mobile chat by significantly reducing the latency in the image selection process (3.19 × improvement) and consequently more frequent image usage (1.8 ×) than existing solutions. Our study participants reported that they used images more often with MilliCat as the images helped them convey information more effectively, emphasize their opinion, express emotions, and have fun chatting experience.

Original languageEnglish
Title of host publicationConference Proceedings - 22nd International Conference on Human-Computer Interaction with Mobile Devices and Services
Subtitle of host publicationExpanding the Horizon of Mobile Interaction, MobileHCI 2020
PublisherAssociation for Computing Machinery, Inc
ISBN (Electronic)9781450375160
DOIs
Publication statusPublished - 2020 Oct 5
Event22nd International Conference on Human-Computer Interaction with Mobile Devices and Services: Expanding the Horizon of Mobile Interaction, MobileHCI 2020 - Virtual, Online, Germany
Duration: 2020 Oct 52020 Oct 9

Publication series

NameConference Proceedings - 22nd International Conference on Human-Computer Interaction with Mobile Devices and Services: Expanding the Horizon of Mobile Interaction, MobileHCI 2020

Conference

Conference22nd International Conference on Human-Computer Interaction with Mobile Devices and Services: Expanding the Horizon of Mobile Interaction, MobileHCI 2020
CountryGermany
CityVirtual, Online
Period20/10/520/10/9

Bibliographical note

Funding Information:
This work was supported in part by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIP) (No. NRF-2020R1A2C1004062), Next-Generation Information Computing Development Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Science and ICT (NRF-2017M3C4A7083534), and the Basic Science Research Program through the National Research Foundation of Korea funded by the Ministry of Science and ICT (2018R1C1B6003869). The authors thank Alvin Chiang, Youngkyu Hong, Young Seok Kim, and Chia-Wei Wu for their early contribution.

All Science Journal Classification (ASJC) codes

  • Computer Networks and Communications
  • Information Systems
  • Human-Computer Interaction
  • Software

Fingerprint Dive into the research topics of 'Messaging beyond texts with real-time image suggestions'. Together they form a unique fingerprint.

Cite this