Hand-free Natural User Interface for VR HMD with IR based facial gesture tracking sensor

Jinhyuk Kim, Jaekwang Cha, Hojun Lee, Shiho Kim

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We proposed a hand-free Natural User Interface (NUI) for VR Head-Mounted-Display (HMD) with infra-red (IR) based sensor tracking facial gestures. We have realized NUI based on the real-time recognition of user intuitions for VR HMD without any additional control devices except for a built-in Gyroscope and IR couplers with readout circuitry integrated in the foam interface of an HMD. We implemented seven control commands affordable for 3D interactions with virtual objects. The experimental data show that the proposed system provides a convenient and efficient 2D/3D user interface for both manipulating objects and controlling commands while wearing a VR HMD.

Original languageEnglish
Title of host publicationProceedings - VRST 2017
Subtitle of host publication23rd ACM Conference on Virtual Reality Software and Technology
EditorsStephen N. Spencer
PublisherAssociation for Computing Machinery
ISBN (Electronic)9781450355483
DOIs
Publication statusPublished - 2017 Nov 8
Event23rd ACM Conference on Virtual Reality Software and Technology, VRST 2017 - Gothenburg, Sweden
Duration: 2017 Nov 82017 Nov 10

Publication series

NameProceedings of the ACM Symposium on Virtual Reality Software and Technology, VRST
VolumePart F131944

Other

Other23rd ACM Conference on Virtual Reality Software and Technology, VRST 2017
CountrySweden
CityGothenburg
Period17/11/817/11/10

Fingerprint

User interfaces
Display devices
Infrared radiation
Sensors
Gyroscopes
Foams

All Science Journal Classification (ASJC) codes

  • Software

Cite this

Kim, J., Cha, J., Lee, H., & Kim, S. (2017). Hand-free Natural User Interface for VR HMD with IR based facial gesture tracking sensor. In S. N. Spencer (Ed.), Proceedings - VRST 2017: 23rd ACM Conference on Virtual Reality Software and Technology [a62] (Proceedings of the ACM Symposium on Virtual Reality Software and Technology, VRST; Vol. Part F131944). Association for Computing Machinery. https://doi.org/10.1145/3139131.3143420
Kim, Jinhyuk ; Cha, Jaekwang ; Lee, Hojun ; Kim, Shiho. / Hand-free Natural User Interface for VR HMD with IR based facial gesture tracking sensor. Proceedings - VRST 2017: 23rd ACM Conference on Virtual Reality Software and Technology. editor / Stephen N. Spencer. Association for Computing Machinery, 2017. (Proceedings of the ACM Symposium on Virtual Reality Software and Technology, VRST).
@inproceedings{09d1582357bc4bb7b9ae7e126d6f2496,
title = "Hand-free Natural User Interface for VR HMD with IR based facial gesture tracking sensor",
abstract = "We proposed a hand-free Natural User Interface (NUI) for VR Head-Mounted-Display (HMD) with infra-red (IR) based sensor tracking facial gestures. We have realized NUI based on the real-time recognition of user intuitions for VR HMD without any additional control devices except for a built-in Gyroscope and IR couplers with readout circuitry integrated in the foam interface of an HMD. We implemented seven control commands affordable for 3D interactions with virtual objects. The experimental data show that the proposed system provides a convenient and efficient 2D/3D user interface for both manipulating objects and controlling commands while wearing a VR HMD.",
author = "Jinhyuk Kim and Jaekwang Cha and Hojun Lee and Shiho Kim",
year = "2017",
month = "11",
day = "8",
doi = "10.1145/3139131.3143420",
language = "English",
series = "Proceedings of the ACM Symposium on Virtual Reality Software and Technology, VRST",
publisher = "Association for Computing Machinery",
editor = "Spencer, {Stephen N.}",
booktitle = "Proceedings - VRST 2017",

}

Kim, J, Cha, J, Lee, H & Kim, S 2017, Hand-free Natural User Interface for VR HMD with IR based facial gesture tracking sensor. in SN Spencer (ed.), Proceedings - VRST 2017: 23rd ACM Conference on Virtual Reality Software and Technology., a62, Proceedings of the ACM Symposium on Virtual Reality Software and Technology, VRST, vol. Part F131944, Association for Computing Machinery, 23rd ACM Conference on Virtual Reality Software and Technology, VRST 2017, Gothenburg, Sweden, 17/11/8. https://doi.org/10.1145/3139131.3143420

Hand-free Natural User Interface for VR HMD with IR based facial gesture tracking sensor. / Kim, Jinhyuk; Cha, Jaekwang; Lee, Hojun; Kim, Shiho.

Proceedings - VRST 2017: 23rd ACM Conference on Virtual Reality Software and Technology. ed. / Stephen N. Spencer. Association for Computing Machinery, 2017. a62 (Proceedings of the ACM Symposium on Virtual Reality Software and Technology, VRST; Vol. Part F131944).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Hand-free Natural User Interface for VR HMD with IR based facial gesture tracking sensor

AU - Kim, Jinhyuk

AU - Cha, Jaekwang

AU - Lee, Hojun

AU - Kim, Shiho

PY - 2017/11/8

Y1 - 2017/11/8

N2 - We proposed a hand-free Natural User Interface (NUI) for VR Head-Mounted-Display (HMD) with infra-red (IR) based sensor tracking facial gestures. We have realized NUI based on the real-time recognition of user intuitions for VR HMD without any additional control devices except for a built-in Gyroscope and IR couplers with readout circuitry integrated in the foam interface of an HMD. We implemented seven control commands affordable for 3D interactions with virtual objects. The experimental data show that the proposed system provides a convenient and efficient 2D/3D user interface for both manipulating objects and controlling commands while wearing a VR HMD.

AB - We proposed a hand-free Natural User Interface (NUI) for VR Head-Mounted-Display (HMD) with infra-red (IR) based sensor tracking facial gestures. We have realized NUI based on the real-time recognition of user intuitions for VR HMD without any additional control devices except for a built-in Gyroscope and IR couplers with readout circuitry integrated in the foam interface of an HMD. We implemented seven control commands affordable for 3D interactions with virtual objects. The experimental data show that the proposed system provides a convenient and efficient 2D/3D user interface for both manipulating objects and controlling commands while wearing a VR HMD.

UR - http://www.scopus.com/inward/record.url?scp=85038574578&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85038574578&partnerID=8YFLogxK

U2 - 10.1145/3139131.3143420

DO - 10.1145/3139131.3143420

M3 - Conference contribution

T3 - Proceedings of the ACM Symposium on Virtual Reality Software and Technology, VRST

BT - Proceedings - VRST 2017

A2 - Spencer, Stephen N.

PB - Association for Computing Machinery

ER -

Kim J, Cha J, Lee H, Kim S. Hand-free Natural User Interface for VR HMD with IR based facial gesture tracking sensor. In Spencer SN, editor, Proceedings - VRST 2017: 23rd ACM Conference on Virtual Reality Software and Technology. Association for Computing Machinery. 2017. a62. (Proceedings of the ACM Symposium on Virtual Reality Software and Technology, VRST). https://doi.org/10.1145/3139131.3143420