An IR-based facial expression tracking sensor for head-mounted displays

Jaekwang Cha, Jinhyuk Kim, Shiho Kim

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We proposed an IR-based facial expression tracking sensor for Head-Mounted Displays(HMD). The proposed sensor uses lateral propagation characteristics of IR light on human skin to capture the degree of compressed or stretched deformations of facial skin. We derived a semi-empirical equation modeling the lateral propagation characteristics of vertically incident IR light into human skin, with parameters which fit to the measured data. All components of IR emitters and receivers were integrated into the form interface of a commercial VR headset. We verified the functionality of tracking performance from 4-kind of basic facial expressions in the experiment using the implemented prototype headset. As a further work, we will upgrade the performance of proposed sensor to recognize emotional expressions of HMD user by applying machine learning technique. We are aiming to enable immersive human to human(or avatar) communications in the cyberspace.

Original languageEnglish
Title of host publicationIEEE Sensors, SENSORS 2016 - Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781479982875
DOIs
Publication statusPublished - 2017 Jan 5
Event15th IEEE Sensors Conference, SENSORS 2016 - Orlando, United States
Duration: 2016 Oct 302016 Nov 2

Other

Other15th IEEE Sensors Conference, SENSORS 2016
CountryUnited States
CityOrlando
Period16/10/3016/11/2

Fingerprint

Skin
Display devices
Sensors
Infrared radiation
Learning systems
Communication
Experiments

All Science Journal Classification (ASJC) codes

  • Electrical and Electronic Engineering

Cite this

Cha, J., Kim, J., & Kim, S. (2017). An IR-based facial expression tracking sensor for head-mounted displays. In IEEE Sensors, SENSORS 2016 - Proceedings [7808913] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICSENS.2016.7808913
Cha, Jaekwang ; Kim, Jinhyuk ; Kim, Shiho. / An IR-based facial expression tracking sensor for head-mounted displays. IEEE Sensors, SENSORS 2016 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2017.
@inproceedings{27ae06fddbbc4c118822559da40c1f3a,
title = "An IR-based facial expression tracking sensor for head-mounted displays",
abstract = "We proposed an IR-based facial expression tracking sensor for Head-Mounted Displays(HMD). The proposed sensor uses lateral propagation characteristics of IR light on human skin to capture the degree of compressed or stretched deformations of facial skin. We derived a semi-empirical equation modeling the lateral propagation characteristics of vertically incident IR light into human skin, with parameters which fit to the measured data. All components of IR emitters and receivers were integrated into the form interface of a commercial VR headset. We verified the functionality of tracking performance from 4-kind of basic facial expressions in the experiment using the implemented prototype headset. As a further work, we will upgrade the performance of proposed sensor to recognize emotional expressions of HMD user by applying machine learning technique. We are aiming to enable immersive human to human(or avatar) communications in the cyberspace.",
author = "Jaekwang Cha and Jinhyuk Kim and Shiho Kim",
year = "2017",
month = "1",
day = "5",
doi = "10.1109/ICSENS.2016.7808913",
language = "English",
booktitle = "IEEE Sensors, SENSORS 2016 - Proceedings",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
address = "United States",

}

Cha, J, Kim, J & Kim, S 2017, An IR-based facial expression tracking sensor for head-mounted displays. in IEEE Sensors, SENSORS 2016 - Proceedings., 7808913, Institute of Electrical and Electronics Engineers Inc., 15th IEEE Sensors Conference, SENSORS 2016, Orlando, United States, 16/10/30. https://doi.org/10.1109/ICSENS.2016.7808913

An IR-based facial expression tracking sensor for head-mounted displays. / Cha, Jaekwang; Kim, Jinhyuk; Kim, Shiho.

IEEE Sensors, SENSORS 2016 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2017. 7808913.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - An IR-based facial expression tracking sensor for head-mounted displays

AU - Cha, Jaekwang

AU - Kim, Jinhyuk

AU - Kim, Shiho

PY - 2017/1/5

Y1 - 2017/1/5

N2 - We proposed an IR-based facial expression tracking sensor for Head-Mounted Displays(HMD). The proposed sensor uses lateral propagation characteristics of IR light on human skin to capture the degree of compressed or stretched deformations of facial skin. We derived a semi-empirical equation modeling the lateral propagation characteristics of vertically incident IR light into human skin, with parameters which fit to the measured data. All components of IR emitters and receivers were integrated into the form interface of a commercial VR headset. We verified the functionality of tracking performance from 4-kind of basic facial expressions in the experiment using the implemented prototype headset. As a further work, we will upgrade the performance of proposed sensor to recognize emotional expressions of HMD user by applying machine learning technique. We are aiming to enable immersive human to human(or avatar) communications in the cyberspace.

AB - We proposed an IR-based facial expression tracking sensor for Head-Mounted Displays(HMD). The proposed sensor uses lateral propagation characteristics of IR light on human skin to capture the degree of compressed or stretched deformations of facial skin. We derived a semi-empirical equation modeling the lateral propagation characteristics of vertically incident IR light into human skin, with parameters which fit to the measured data. All components of IR emitters and receivers were integrated into the form interface of a commercial VR headset. We verified the functionality of tracking performance from 4-kind of basic facial expressions in the experiment using the implemented prototype headset. As a further work, we will upgrade the performance of proposed sensor to recognize emotional expressions of HMD user by applying machine learning technique. We are aiming to enable immersive human to human(or avatar) communications in the cyberspace.

UR - http://www.scopus.com/inward/record.url?scp=85010961035&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85010961035&partnerID=8YFLogxK

U2 - 10.1109/ICSENS.2016.7808913

DO - 10.1109/ICSENS.2016.7808913

M3 - Conference contribution

AN - SCOPUS:85010961035

BT - IEEE Sensors, SENSORS 2016 - Proceedings

PB - Institute of Electrical and Electronics Engineers Inc.

ER -

Cha J, Kim J, Kim S. An IR-based facial expression tracking sensor for head-mounted displays. In IEEE Sensors, SENSORS 2016 - Proceedings. Institute of Electrical and Electronics Engineers Inc. 2017. 7808913 https://doi.org/10.1109/ICSENS.2016.7808913