A deep learning-based stress detection algorithm with speech signal

Hyewon Han, Kyunggeun Byun, Hong-Goo Kang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

In this paper, we propose a deep learning-based psychological stress detection algorithm using speech signals. With increasing demands for communication between human and intelligent systems, automatic stress detection is becoming an interesting research topic. Stress can be reliably detected by measuring the level of specific hormones (e.g., cortisol), but this is not a convenient method for the detection of stress in human-machine interactions. The proposed algorithm first extracts mel-filterbank coefficients using preprocessed speech data and then predicts the status of stress output using a binary decision criterion (i.e., stressed or unstressed) using long short-term memory (LSTM) and feed-forward networks. To evaluate the performance of the proposed algorithm, speech, video, and bio-signal data were collected in a well-controlled environment. We utilized only speech signals in the decision process from subjects whose salivary cortisol level varies over 10%. Using the proposed algorithm, we achieved 66.4% accuracy in detecting the stress state from 25 subjects, thereby demonstrating the possibility of utilizing speech signals for automatic stress detection.

Original languageEnglish
Title of host publicationAVSU 2018 - Proceedings of the 2018 Workshop on Audio-Visual Scene Understanding for Immersive Multimedia, Co-located with MM 2018
PublisherAssociation for Computing Machinery, Inc
Pages11-15
Number of pages5
ISBN (Electronic)9781450359771
DOIs
Publication statusPublished - 2018 Oct 26
Event2018 Workshop on Audio-Visual Scene Understanding for Immersive Multimedia, AVSU 2018, co-located with MM 2018 - Seoul, Korea, Republic of
Duration: 2018 Oct 26 → …

Publication series

NameAVSU 2018 - Proceedings of the 2018 Workshop on Audio-Visual Scene Understanding for Immersive Multimedia, Co-located with MM 2018

Conference

Conference2018 Workshop on Audio-Visual Scene Understanding for Immersive Multimedia, AVSU 2018, co-located with MM 2018
CountryKorea, Republic of
CitySeoul
Period18/10/26 → …

Fingerprint

Cortisol
Hormones
Intelligent systems
Deep learning
Communication
Long short-term memory

All Science Journal Classification (ASJC) codes

  • Software
  • Media Technology
  • Computer Graphics and Computer-Aided Design
  • Computer Vision and Pattern Recognition

Cite this

Han, H., Byun, K., & Kang, H-G. (2018). A deep learning-based stress detection algorithm with speech signal. In AVSU 2018 - Proceedings of the 2018 Workshop on Audio-Visual Scene Understanding for Immersive Multimedia, Co-located with MM 2018 (pp. 11-15). (AVSU 2018 - Proceedings of the 2018 Workshop on Audio-Visual Scene Understanding for Immersive Multimedia, Co-located with MM 2018). Association for Computing Machinery, Inc. https://doi.org/10.1145/3264869.3264875
Han, Hyewon ; Byun, Kyunggeun ; Kang, Hong-Goo. / A deep learning-based stress detection algorithm with speech signal. AVSU 2018 - Proceedings of the 2018 Workshop on Audio-Visual Scene Understanding for Immersive Multimedia, Co-located with MM 2018. Association for Computing Machinery, Inc, 2018. pp. 11-15 (AVSU 2018 - Proceedings of the 2018 Workshop on Audio-Visual Scene Understanding for Immersive Multimedia, Co-located with MM 2018).
@inproceedings{82023b6306b548fd887de6889e14b5ef,
title = "A deep learning-based stress detection algorithm with speech signal",
abstract = "In this paper, we propose a deep learning-based psychological stress detection algorithm using speech signals. With increasing demands for communication between human and intelligent systems, automatic stress detection is becoming an interesting research topic. Stress can be reliably detected by measuring the level of specific hormones (e.g., cortisol), but this is not a convenient method for the detection of stress in human-machine interactions. The proposed algorithm first extracts mel-filterbank coefficients using preprocessed speech data and then predicts the status of stress output using a binary decision criterion (i.e., stressed or unstressed) using long short-term memory (LSTM) and feed-forward networks. To evaluate the performance of the proposed algorithm, speech, video, and bio-signal data were collected in a well-controlled environment. We utilized only speech signals in the decision process from subjects whose salivary cortisol level varies over 10{\%}. Using the proposed algorithm, we achieved 66.4{\%} accuracy in detecting the stress state from 25 subjects, thereby demonstrating the possibility of utilizing speech signals for automatic stress detection.",
author = "Hyewon Han and Kyunggeun Byun and Hong-Goo Kang",
year = "2018",
month = "10",
day = "26",
doi = "10.1145/3264869.3264875",
language = "English",
series = "AVSU 2018 - Proceedings of the 2018 Workshop on Audio-Visual Scene Understanding for Immersive Multimedia, Co-located with MM 2018",
publisher = "Association for Computing Machinery, Inc",
pages = "11--15",
booktitle = "AVSU 2018 - Proceedings of the 2018 Workshop on Audio-Visual Scene Understanding for Immersive Multimedia, Co-located with MM 2018",

}

Han, H, Byun, K & Kang, H-G 2018, A deep learning-based stress detection algorithm with speech signal. in AVSU 2018 - Proceedings of the 2018 Workshop on Audio-Visual Scene Understanding for Immersive Multimedia, Co-located with MM 2018. AVSU 2018 - Proceedings of the 2018 Workshop on Audio-Visual Scene Understanding for Immersive Multimedia, Co-located with MM 2018, Association for Computing Machinery, Inc, pp. 11-15, 2018 Workshop on Audio-Visual Scene Understanding for Immersive Multimedia, AVSU 2018, co-located with MM 2018, Seoul, Korea, Republic of, 18/10/26. https://doi.org/10.1145/3264869.3264875

A deep learning-based stress detection algorithm with speech signal. / Han, Hyewon; Byun, Kyunggeun; Kang, Hong-Goo.

AVSU 2018 - Proceedings of the 2018 Workshop on Audio-Visual Scene Understanding for Immersive Multimedia, Co-located with MM 2018. Association for Computing Machinery, Inc, 2018. p. 11-15 (AVSU 2018 - Proceedings of the 2018 Workshop on Audio-Visual Scene Understanding for Immersive Multimedia, Co-located with MM 2018).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - A deep learning-based stress detection algorithm with speech signal

AU - Han, Hyewon

AU - Byun, Kyunggeun

AU - Kang, Hong-Goo

PY - 2018/10/26

Y1 - 2018/10/26

N2 - In this paper, we propose a deep learning-based psychological stress detection algorithm using speech signals. With increasing demands for communication between human and intelligent systems, automatic stress detection is becoming an interesting research topic. Stress can be reliably detected by measuring the level of specific hormones (e.g., cortisol), but this is not a convenient method for the detection of stress in human-machine interactions. The proposed algorithm first extracts mel-filterbank coefficients using preprocessed speech data and then predicts the status of stress output using a binary decision criterion (i.e., stressed or unstressed) using long short-term memory (LSTM) and feed-forward networks. To evaluate the performance of the proposed algorithm, speech, video, and bio-signal data were collected in a well-controlled environment. We utilized only speech signals in the decision process from subjects whose salivary cortisol level varies over 10%. Using the proposed algorithm, we achieved 66.4% accuracy in detecting the stress state from 25 subjects, thereby demonstrating the possibility of utilizing speech signals for automatic stress detection.

AB - In this paper, we propose a deep learning-based psychological stress detection algorithm using speech signals. With increasing demands for communication between human and intelligent systems, automatic stress detection is becoming an interesting research topic. Stress can be reliably detected by measuring the level of specific hormones (e.g., cortisol), but this is not a convenient method for the detection of stress in human-machine interactions. The proposed algorithm first extracts mel-filterbank coefficients using preprocessed speech data and then predicts the status of stress output using a binary decision criterion (i.e., stressed or unstressed) using long short-term memory (LSTM) and feed-forward networks. To evaluate the performance of the proposed algorithm, speech, video, and bio-signal data were collected in a well-controlled environment. We utilized only speech signals in the decision process from subjects whose salivary cortisol level varies over 10%. Using the proposed algorithm, we achieved 66.4% accuracy in detecting the stress state from 25 subjects, thereby demonstrating the possibility of utilizing speech signals for automatic stress detection.

UR - http://www.scopus.com/inward/record.url?scp=85061791967&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85061791967&partnerID=8YFLogxK

U2 - 10.1145/3264869.3264875

DO - 10.1145/3264869.3264875

M3 - Conference contribution

AN - SCOPUS:85061791967

T3 - AVSU 2018 - Proceedings of the 2018 Workshop on Audio-Visual Scene Understanding for Immersive Multimedia, Co-located with MM 2018

SP - 11

EP - 15

BT - AVSU 2018 - Proceedings of the 2018 Workshop on Audio-Visual Scene Understanding for Immersive Multimedia, Co-located with MM 2018

PB - Association for Computing Machinery, Inc

ER -

Han H, Byun K, Kang H-G. A deep learning-based stress detection algorithm with speech signal. In AVSU 2018 - Proceedings of the 2018 Workshop on Audio-Visual Scene Understanding for Immersive Multimedia, Co-located with MM 2018. Association for Computing Machinery, Inc. 2018. p. 11-15. (AVSU 2018 - Proceedings of the 2018 Workshop on Audio-Visual Scene Understanding for Immersive Multimedia, Co-located with MM 2018). https://doi.org/10.1145/3264869.3264875