A deep learning-based stress detection algorithm with speech signal

Hyewon Han; Kyunggeun Byun; Hong Goo Kang

doi:10.1145/3264869.3264875

A deep learning-based stress detection algorithm with speech signal

Hyewon Han, Kyunggeun Byun, Hong Goo Kang

Department of Electrical and Electronic Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

14 Citations (Scopus)

Abstract

In this paper, we propose a deep learning-based psychological stress detection algorithm using speech signals. With increasing demands for communication between human and intelligent systems, automatic stress detection is becoming an interesting research topic. Stress can be reliably detected by measuring the level of specific hormones (e.g., cortisol), but this is not a convenient method for the detection of stress in human-machine interactions. The proposed algorithm first extracts mel-filterbank coefficients using preprocessed speech data and then predicts the status of stress output using a binary decision criterion (i.e., stressed or unstressed) using long short-term memory (LSTM) and feed-forward networks. To evaluate the performance of the proposed algorithm, speech, video, and bio-signal data were collected in a well-controlled environment. We utilized only speech signals in the decision process from subjects whose salivary cortisol level varies over 10%. Using the proposed algorithm, we achieved 66.4% accuracy in detecting the stress state from 25 subjects, thereby demonstrating the possibility of utilizing speech signals for automatic stress detection.

Original language	English
Title of host publication	AVSU 2018 - Proceedings of the 2018 Workshop on Audio-Visual Scene Understanding for Immersive Multimedia, Co-located with MM 2018
Publisher	Association for Computing Machinery, Inc
Pages	11-15
Number of pages	5
ISBN (Electronic)	9781450359771
DOIs	https://doi.org/10.1145/3264869.3264875
Publication status	Published - 2018 Oct 26
Event	2018 Workshop on Audio-Visual Scene Understanding for Immersive Multimedia, AVSU 2018, co-located with MM 2018 - Seoul, Korea, Republic of Duration: 2018 Oct 26 → …

Publication series

Name	AVSU 2018 - Proceedings of the 2018 Workshop on Audio-Visual Scene Understanding for Immersive Multimedia, Co-located with MM 2018

Conference

Conference	2018 Workshop on Audio-Visual Scene Understanding for Immersive Multimedia, AVSU 2018, co-located with MM 2018
Country/Territory	Korea, Republic of
City	Seoul
Period	18/10/26 → …

Bibliographical note

Funding Information:
The work is supported by the National Research Foundation of Korea. under Grant No.: 2016-0-00562 and Korean Flagship Project on AI (http://www.nrf.re.kr). vfill

Publisher Copyright:
© 2018 Association for Computing Machinery.

All Science Journal Classification (ASJC) codes

Software
Media Technology
Computer Graphics and Computer-Aided Design
Computer Vision and Pattern Recognition

Access to Document

10.1145/3264869.3264875

Cite this

Han, H., Byun, K., & Kang, H. G. (2018). A deep learning-based stress detection algorithm with speech signal. In AVSU 2018 - Proceedings of the 2018 Workshop on Audio-Visual Scene Understanding for Immersive Multimedia, Co-located with MM 2018 (pp. 11-15). (AVSU 2018 - Proceedings of the 2018 Workshop on Audio-Visual Scene Understanding for Immersive Multimedia, Co-located with MM 2018). Association for Computing Machinery, Inc. https://doi.org/10.1145/3264869.3264875

Han, Hyewon ; Byun, Kyunggeun ; Kang, Hong Goo. / A deep learning-based stress detection algorithm with speech signal. AVSU 2018 - Proceedings of the 2018 Workshop on Audio-Visual Scene Understanding for Immersive Multimedia, Co-located with MM 2018. Association for Computing Machinery, Inc, 2018. pp. 11-15 (AVSU 2018 - Proceedings of the 2018 Workshop on Audio-Visual Scene Understanding for Immersive Multimedia, Co-located with MM 2018).

@inproceedings{82023b6306b548fd887de6889e14b5ef,

title = "A deep learning-based stress detection algorithm with speech signal",

abstract = "In this paper, we propose a deep learning-based psychological stress detection algorithm using speech signals. With increasing demands for communication between human and intelligent systems, automatic stress detection is becoming an interesting research topic. Stress can be reliably detected by measuring the level of specific hormones (e.g., cortisol), but this is not a convenient method for the detection of stress in human-machine interactions. The proposed algorithm first extracts mel-filterbank coefficients using preprocessed speech data and then predicts the status of stress output using a binary decision criterion (i.e., stressed or unstressed) using long short-term memory (LSTM) and feed-forward networks. To evaluate the performance of the proposed algorithm, speech, video, and bio-signal data were collected in a well-controlled environment. We utilized only speech signals in the decision process from subjects whose salivary cortisol level varies over 10%. Using the proposed algorithm, we achieved 66.4% accuracy in detecting the stress state from 25 subjects, thereby demonstrating the possibility of utilizing speech signals for automatic stress detection.",

author = "Hyewon Han and Kyunggeun Byun and Kang, {Hong Goo}",

note = "Funding Information: The work is supported by the National Research Foundation of Korea. under Grant No.: 2016-0-00562 and Korean Flagship Project on AI (http://www.nrf.re.kr). vfill Publisher Copyright: {\textcopyright} 2018 Association for Computing Machinery.; 2018 Workshop on Audio-Visual Scene Understanding for Immersive Multimedia, AVSU 2018, co-located with MM 2018 ; Conference date: 26-10-2018",

year = "2018",

month = oct,

day = "26",

doi = "10.1145/3264869.3264875",

language = "English",

series = "AVSU 2018 - Proceedings of the 2018 Workshop on Audio-Visual Scene Understanding for Immersive Multimedia, Co-located with MM 2018",

publisher = "Association for Computing Machinery, Inc",

pages = "11--15",

booktitle = "AVSU 2018 - Proceedings of the 2018 Workshop on Audio-Visual Scene Understanding for Immersive Multimedia, Co-located with MM 2018",

}

Han, H, Byun, K & Kang, HG 2018, A deep learning-based stress detection algorithm with speech signal. in AVSU 2018 - Proceedings of the 2018 Workshop on Audio-Visual Scene Understanding for Immersive Multimedia, Co-located with MM 2018. AVSU 2018 - Proceedings of the 2018 Workshop on Audio-Visual Scene Understanding for Immersive Multimedia, Co-located with MM 2018, Association for Computing Machinery, Inc, pp. 11-15, 2018 Workshop on Audio-Visual Scene Understanding for Immersive Multimedia, AVSU 2018, co-located with MM 2018, Seoul, Korea, Republic of, 18/10/26. https://doi.org/10.1145/3264869.3264875

A deep learning-based stress detection algorithm with speech signal. / Han, Hyewon; Byun, Kyunggeun; Kang, Hong Goo.

AVSU 2018 - Proceedings of the 2018 Workshop on Audio-Visual Scene Understanding for Immersive Multimedia, Co-located with MM 2018. Association for Computing Machinery, Inc, 2018. p. 11-15 (AVSU 2018 - Proceedings of the 2018 Workshop on Audio-Visual Scene Understanding for Immersive Multimedia, Co-located with MM 2018).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - A deep learning-based stress detection algorithm with speech signal

AU - Han, Hyewon

AU - Byun, Kyunggeun

AU - Kang, Hong Goo

N1 - Funding Information: The work is supported by the National Research Foundation of Korea. under Grant No.: 2016-0-00562 and Korean Flagship Project on AI (http://www.nrf.re.kr). vfill Publisher Copyright: © 2018 Association for Computing Machinery.

PY - 2018/10/26

Y1 - 2018/10/26

N2 - In this paper, we propose a deep learning-based psychological stress detection algorithm using speech signals. With increasing demands for communication between human and intelligent systems, automatic stress detection is becoming an interesting research topic. Stress can be reliably detected by measuring the level of specific hormones (e.g., cortisol), but this is not a convenient method for the detection of stress in human-machine interactions. The proposed algorithm first extracts mel-filterbank coefficients using preprocessed speech data and then predicts the status of stress output using a binary decision criterion (i.e., stressed or unstressed) using long short-term memory (LSTM) and feed-forward networks. To evaluate the performance of the proposed algorithm, speech, video, and bio-signal data were collected in a well-controlled environment. We utilized only speech signals in the decision process from subjects whose salivary cortisol level varies over 10%. Using the proposed algorithm, we achieved 66.4% accuracy in detecting the stress state from 25 subjects, thereby demonstrating the possibility of utilizing speech signals for automatic stress detection.

AB - In this paper, we propose a deep learning-based psychological stress detection algorithm using speech signals. With increasing demands for communication between human and intelligent systems, automatic stress detection is becoming an interesting research topic. Stress can be reliably detected by measuring the level of specific hormones (e.g., cortisol), but this is not a convenient method for the detection of stress in human-machine interactions. The proposed algorithm first extracts mel-filterbank coefficients using preprocessed speech data and then predicts the status of stress output using a binary decision criterion (i.e., stressed or unstressed) using long short-term memory (LSTM) and feed-forward networks. To evaluate the performance of the proposed algorithm, speech, video, and bio-signal data were collected in a well-controlled environment. We utilized only speech signals in the decision process from subjects whose salivary cortisol level varies over 10%. Using the proposed algorithm, we achieved 66.4% accuracy in detecting the stress state from 25 subjects, thereby demonstrating the possibility of utilizing speech signals for automatic stress detection.

UR - http://www.scopus.com/inward/record.url?scp=85061791967&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85061791967&partnerID=8YFLogxK

U2 - 10.1145/3264869.3264875

DO - 10.1145/3264869.3264875

M3 - Conference contribution

AN - SCOPUS:85061791967

T3 - AVSU 2018 - Proceedings of the 2018 Workshop on Audio-Visual Scene Understanding for Immersive Multimedia, Co-located with MM 2018

SP - 11

EP - 15

BT - AVSU 2018 - Proceedings of the 2018 Workshop on Audio-Visual Scene Understanding for Immersive Multimedia, Co-located with MM 2018

PB - Association for Computing Machinery, Inc

T2 - 2018 Workshop on Audio-Visual Scene Understanding for Immersive Multimedia, AVSU 2018, co-located with MM 2018

Y2 - 26 October 2018

ER -

Han H, Byun K, Kang HG. A deep learning-based stress detection algorithm with speech signal. In AVSU 2018 - Proceedings of the 2018 Workshop on Audio-Visual Scene Understanding for Immersive Multimedia, Co-located with MM 2018. Association for Computing Machinery, Inc. 2018. p. 11-15. (AVSU 2018 - Proceedings of the 2018 Workshop on Audio-Visual Scene Understanding for Immersive Multimedia, Co-located with MM 2018). https://doi.org/10.1145/3264869.3264875

A deep learning-based stress detection algorithm with speech signal

Abstract

Publication series

Conference

Bibliographical note

All Science Journal Classification (ASJC) codes

Access to Document

Other files and links

Fingerprint

Cite this