HCI Based In-Cabin Monitoring System for Irregular Situations with Occupants Facial Anonymization

Ashutosh Mishra, Jaekwang Cha, Shiho Kim

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Citations (Scopus)


Absence of human in-charge in fully autonomous vehicles (FAVs) impose multi-pronged in-cabin monitoring in real-time to monitor any irregular situation. Moreover, occupant’s visual information (such as video, images) transmission from in-cabin of vehicle to the data center and control-room is required for various in-cabin monitoring tasks. However, this information transfer may cause a substantial threat to the occupants. Though applying some patch on face of the occupants protects personal information however, simultaneously it deteriorates the important facial information which is very crucial in monitoring various tasks such as emotion detection. Therefore, a human-computer-interaction (HCI) is required to manage this trade-off between facial information and facial anonymization without any important information loss. Also, this HCI based in-cabin monitoring system should be implicit to retain the users trust in the intelligent transportation system (ITS). In this paper, we have proposed generative adversarial network (GAN) based in-cabin monitoring approach for emotion detection in FAV. In this method, we proposed to generate an artificial (virtual) face having real-facial expressions to provide facial anonymity to the occupants while monitoring FAV cabin. Therefore, this method provides anonymous facial information which is essentially required in detection and monitoring tasks to avoid irregular situations. We have used both publicly available and our own in-cabin dataset to assess our proposed approach. Our experiments show satisfactory results.

Original languageEnglish
Title of host publicationIntelligent Human Computer Interaction - 12th International Conference, IHCI 2020, Proceedings
EditorsMadhusudan Singh, Dae-Ki Kang, Jong-Ha Lee, Uma Shanker Tiwary, Dhananjay Singh, Wan-Young Chung
PublisherSpringer Science and Business Media Deutschland GmbH
Number of pages11
ISBN (Print)9783030684518
Publication statusPublished - 2021
Event12th International Conference on Intelligent Human Computer Interaction, IHCI 2020 - Daegu, Korea, Republic of
Duration: 2020 Nov 242020 Nov 26

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume12616 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


Conference12th International Conference on Intelligent Human Computer Interaction, IHCI 2020
Country/TerritoryKorea, Republic of

Bibliographical note

Funding Information:
Acknowledgment. This research was supported by Korea Research Fellowship program funded by the Ministry of Science and ICT through the National Research Foundation of Korea (NRF-2019H1D3A1A01071115).

Publisher Copyright:
© 2021, Springer Nature Switzerland AG.

All Science Journal Classification (ASJC) codes

  • Theoretical Computer Science
  • Computer Science(all)


Dive into the research topics of 'HCI Based In-Cabin Monitoring System for Irregular Situations with Occupants Facial Anonymization'. Together they form a unique fingerprint.

Cite this