Modeling immersive media experiences by sensing impact on subjects

Eleni Kroupi, Philippe Hanhart, Jong-Seok Lee, Martin Rerabek, Touradj Ebrahimi

Research output: Contribution to journalArticle

5 Citations (Scopus)

Abstract

As immersive technologies target to provide higher quality of multimedia experiences, it is important to understand the quality of experience (QoE) perceived by users from various multimedia rendering schemes, in order to design and optimize human-centric immersive multimedia systems. In this study, various QoE-related aspects, such as depth perception, sensation of reality, content preference, and perceived quality are investigated and compared for presentation of 2D and 3D contents. Since the advantages of implicit over explicit QoE assessment have become essential, the way these QoE-related aspects influence brain and periphery is also investigated. In particular, two classification schemes using electroencephalography (EEG) and peripheral signals (electrocardiography and respiration) are carried out, to explore if it is possible to automatically recognize the QoE-related aspects under investigation. In addition, a decision-fusion scheme is applied to EEG and peripheral features, to explore the advantage of integrating information from the two modalities. The results reveal that the highest monomodal average informedness is achieved in the high beta EEG band (0.14 % ± 0.09, p < 0.01), when recognizing sensation of reality. The highest and significantly non-random multimodal average informedness is achieved when high beta EEG band is fused with peripheral features (0.17 % ± 0.1, p < 0.01), for the case of sensation of reality. Finally, a temporal analysis is conducted to explore how the EEG correlates for the case of sensation of reality change over time. The results reveal that the right cortex is more involved when sensation of reality is low, and the left when sensation of reality is high, indicating that approach and withdrawal-related processes occur during sensation of reality.

Original languageEnglish
Pages (from-to)12409-12429
Number of pages21
JournalMultimedia Tools and Applications
Volume75
Issue number20
DOIs
Publication statusPublished - 2016 Oct 1

Fingerprint

Electroencephalography
Depth perception
Multimedia systems
Electrocardiography
Brain
Fusion reactions

All Science Journal Classification (ASJC) codes

  • Software
  • Media Technology
  • Hardware and Architecture
  • Computer Networks and Communications

Cite this

Kroupi, Eleni ; Hanhart, Philippe ; Lee, Jong-Seok ; Rerabek, Martin ; Ebrahimi, Touradj. / Modeling immersive media experiences by sensing impact on subjects. In: Multimedia Tools and Applications. 2016 ; Vol. 75, No. 20. pp. 12409-12429.
@article{a8a78839771d44399253042b88c8e52a,
title = "Modeling immersive media experiences by sensing impact on subjects",
abstract = "As immersive technologies target to provide higher quality of multimedia experiences, it is important to understand the quality of experience (QoE) perceived by users from various multimedia rendering schemes, in order to design and optimize human-centric immersive multimedia systems. In this study, various QoE-related aspects, such as depth perception, sensation of reality, content preference, and perceived quality are investigated and compared for presentation of 2D and 3D contents. Since the advantages of implicit over explicit QoE assessment have become essential, the way these QoE-related aspects influence brain and periphery is also investigated. In particular, two classification schemes using electroencephalography (EEG) and peripheral signals (electrocardiography and respiration) are carried out, to explore if it is possible to automatically recognize the QoE-related aspects under investigation. In addition, a decision-fusion scheme is applied to EEG and peripheral features, to explore the advantage of integrating information from the two modalities. The results reveal that the highest monomodal average informedness is achieved in the high beta EEG band (0.14 {\%} ± 0.09, p < 0.01), when recognizing sensation of reality. The highest and significantly non-random multimodal average informedness is achieved when high beta EEG band is fused with peripheral features (0.17 {\%} ± 0.1, p < 0.01), for the case of sensation of reality. Finally, a temporal analysis is conducted to explore how the EEG correlates for the case of sensation of reality change over time. The results reveal that the right cortex is more involved when sensation of reality is low, and the left when sensation of reality is high, indicating that approach and withdrawal-related processes occur during sensation of reality.",
author = "Eleni Kroupi and Philippe Hanhart and Jong-Seok Lee and Martin Rerabek and Touradj Ebrahimi",
year = "2016",
month = "10",
day = "1",
doi = "10.1007/s11042-015-2980-z",
language = "English",
volume = "75",
pages = "12409--12429",
journal = "Multimedia Tools and Applications",
issn = "1380-7501",
publisher = "Springer Netherlands",
number = "20",

}

Modeling immersive media experiences by sensing impact on subjects. / Kroupi, Eleni; Hanhart, Philippe; Lee, Jong-Seok; Rerabek, Martin; Ebrahimi, Touradj.

In: Multimedia Tools and Applications, Vol. 75, No. 20, 01.10.2016, p. 12409-12429.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Modeling immersive media experiences by sensing impact on subjects

AU - Kroupi, Eleni

AU - Hanhart, Philippe

AU - Lee, Jong-Seok

AU - Rerabek, Martin

AU - Ebrahimi, Touradj

PY - 2016/10/1

Y1 - 2016/10/1

N2 - As immersive technologies target to provide higher quality of multimedia experiences, it is important to understand the quality of experience (QoE) perceived by users from various multimedia rendering schemes, in order to design and optimize human-centric immersive multimedia systems. In this study, various QoE-related aspects, such as depth perception, sensation of reality, content preference, and perceived quality are investigated and compared for presentation of 2D and 3D contents. Since the advantages of implicit over explicit QoE assessment have become essential, the way these QoE-related aspects influence brain and periphery is also investigated. In particular, two classification schemes using electroencephalography (EEG) and peripheral signals (electrocardiography and respiration) are carried out, to explore if it is possible to automatically recognize the QoE-related aspects under investigation. In addition, a decision-fusion scheme is applied to EEG and peripheral features, to explore the advantage of integrating information from the two modalities. The results reveal that the highest monomodal average informedness is achieved in the high beta EEG band (0.14 % ± 0.09, p < 0.01), when recognizing sensation of reality. The highest and significantly non-random multimodal average informedness is achieved when high beta EEG band is fused with peripheral features (0.17 % ± 0.1, p < 0.01), for the case of sensation of reality. Finally, a temporal analysis is conducted to explore how the EEG correlates for the case of sensation of reality change over time. The results reveal that the right cortex is more involved when sensation of reality is low, and the left when sensation of reality is high, indicating that approach and withdrawal-related processes occur during sensation of reality.

AB - As immersive technologies target to provide higher quality of multimedia experiences, it is important to understand the quality of experience (QoE) perceived by users from various multimedia rendering schemes, in order to design and optimize human-centric immersive multimedia systems. In this study, various QoE-related aspects, such as depth perception, sensation of reality, content preference, and perceived quality are investigated and compared for presentation of 2D and 3D contents. Since the advantages of implicit over explicit QoE assessment have become essential, the way these QoE-related aspects influence brain and periphery is also investigated. In particular, two classification schemes using electroencephalography (EEG) and peripheral signals (electrocardiography and respiration) are carried out, to explore if it is possible to automatically recognize the QoE-related aspects under investigation. In addition, a decision-fusion scheme is applied to EEG and peripheral features, to explore the advantage of integrating information from the two modalities. The results reveal that the highest monomodal average informedness is achieved in the high beta EEG band (0.14 % ± 0.09, p < 0.01), when recognizing sensation of reality. The highest and significantly non-random multimodal average informedness is achieved when high beta EEG band is fused with peripheral features (0.17 % ± 0.1, p < 0.01), for the case of sensation of reality. Finally, a temporal analysis is conducted to explore how the EEG correlates for the case of sensation of reality change over time. The results reveal that the right cortex is more involved when sensation of reality is low, and the left when sensation of reality is high, indicating that approach and withdrawal-related processes occur during sensation of reality.

UR - http://www.scopus.com/inward/record.url?scp=84944707207&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84944707207&partnerID=8YFLogxK

U2 - 10.1007/s11042-015-2980-z

DO - 10.1007/s11042-015-2980-z

M3 - Article

VL - 75

SP - 12409

EP - 12429

JO - Multimedia Tools and Applications

JF - Multimedia Tools and Applications

SN - 1380-7501

IS - 20

ER -