Perceptual quality and modeling accuracy of excitation parameters in DLSTM-based speech synthesis systems

Eunwoo Song; Frank K. Soong; Hong Goo Kang

doi:10.1109/ASRU.2017.8269001

Perceptual quality and modeling accuracy of excitation parameters in DLSTM-based speech synthesis systems

Eunwoo Song, Frank K. Soong, Hong Goo Kang

Department of Electrical and Electronic Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

1 Citation (Scopus)

Abstract

This paper investigates how the perceptual quality of the synthesized speech is affected by reconstruction errors in excitation signals generated by a deep learning-based statistical model. In this framework, the excitation signal obtained by an LPC inverse filter is first decomposed into harmonic and noise components using an improved time-frequency trajectory excitation (ITFTE) scheme, then they are trained and generated by a deep long short-term memory (DLSTM)-based speech synthesis system. By controlling the parametric dimension of the ITFTE vocoder, we analyze the impact of the harmonic and noise components to the perceptual quality of the synthesized speech. Both objective and subjective experimental results confirm that the maximum perceptually allowable spectral distortion for the harmonic spectrum of the generated excitation is ∼0.08 dB. On the other hand, the absolute spectral distortion in the noise components is meaningless, and only the spectral envelope is relevant to the perceptual quality.

Original language	English
Title of host publication	2017 IEEE Automatic Speech Recognition and Understanding Workshop, ASRU 2017 - Proceedings
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	671-676
Number of pages	6
ISBN (Electronic)	9781509047888
DOIs	https://doi.org/10.1109/ASRU.2017.8269001
Publication status	Published - 2018 Jan 24
Event	2017 IEEE Automatic Speech Recognition and Understanding Workshop, ASRU 2017 - Okinawa, Japan Duration: 2017 Dec 16 → 2017 Dec 20

Publication series

Name	2017 IEEE Automatic Speech Recognition and Understanding Workshop, ASRU 2017 - Proceedings
Volume	2018-January

Other

Other	2017 IEEE Automatic Speech Recognition and Understanding Workshop, ASRU 2017
Country/Territory	Japan
City	Okinawa
Period	17/12/16 → 17/12/20

Bibliographical note

Funding Information:
This research was supported by Microsoft Research and the MSIP (The Ministry of Science, ICT and Future Planning), Korea, under ICT/SW Creative research program supervised by the IITP (Institute for Information & Communications Technology Promotion). The authors would like to thank Feng-Long Xie, Microsoft Research Asia, Beijing, China, for conducting the listening tests.

Publisher Copyright:
© 2017 IEEE.

All Science Journal Classification (ASJC) codes

Computer Vision and Pattern Recognition
Human-Computer Interaction

Access to Document

10.1109/ASRU.2017.8269001

Cite this

Song, E., Soong, F. K., & Kang, H. G. (2018). Perceptual quality and modeling accuracy of excitation parameters in DLSTM-based speech synthesis systems. In 2017 IEEE Automatic Speech Recognition and Understanding Workshop, ASRU 2017 - Proceedings (pp. 671-676). (2017 IEEE Automatic Speech Recognition and Understanding Workshop, ASRU 2017 - Proceedings; Vol. 2018-January). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ASRU.2017.8269001

Song, Eunwoo ; Soong, Frank K. ; Kang, Hong Goo. / Perceptual quality and modeling accuracy of excitation parameters in DLSTM-based speech synthesis systems. 2017 IEEE Automatic Speech Recognition and Understanding Workshop, ASRU 2017 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2018. pp. 671-676 (2017 IEEE Automatic Speech Recognition and Understanding Workshop, ASRU 2017 - Proceedings).

@inproceedings{5015e312fc734617be8ab1b18ac289d7,

title = "Perceptual quality and modeling accuracy of excitation parameters in DLSTM-based speech synthesis systems",

abstract = "This paper investigates how the perceptual quality of the synthesized speech is affected by reconstruction errors in excitation signals generated by a deep learning-based statistical model. In this framework, the excitation signal obtained by an LPC inverse filter is first decomposed into harmonic and noise components using an improved time-frequency trajectory excitation (ITFTE) scheme, then they are trained and generated by a deep long short-term memory (DLSTM)-based speech synthesis system. By controlling the parametric dimension of the ITFTE vocoder, we analyze the impact of the harmonic and noise components to the perceptual quality of the synthesized speech. Both objective and subjective experimental results confirm that the maximum perceptually allowable spectral distortion for the harmonic spectrum of the generated excitation is ∼0.08 dB. On the other hand, the absolute spectral distortion in the noise components is meaningless, and only the spectral envelope is relevant to the perceptual quality.",

author = "Eunwoo Song and Soong, {Frank K.} and Kang, {Hong Goo}",

note = "Funding Information: This research was supported by Microsoft Research and the MSIP (The Ministry of Science, ICT and Future Planning), Korea, under ICT/SW Creative research program supervised by the IITP (Institute for Information & Communications Technology Promotion). The authors would like to thank Feng-Long Xie, Microsoft Research Asia, Beijing, China, for conducting the listening tests. Publisher Copyright: {\textcopyright} 2017 IEEE.; 2017 IEEE Automatic Speech Recognition and Understanding Workshop, ASRU 2017 ; Conference date: 16-12-2017 Through 20-12-2017",

year = "2018",

month = jan,

day = "24",

doi = "10.1109/ASRU.2017.8269001",

language = "English",

series = "2017 IEEE Automatic Speech Recognition and Understanding Workshop, ASRU 2017 - Proceedings",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "671--676",

booktitle = "2017 IEEE Automatic Speech Recognition and Understanding Workshop, ASRU 2017 - Proceedings",

address = "United States",

}

Song, E, Soong, FK & Kang, HG 2018, Perceptual quality and modeling accuracy of excitation parameters in DLSTM-based speech synthesis systems. in 2017 IEEE Automatic Speech Recognition and Understanding Workshop, ASRU 2017 - Proceedings. 2017 IEEE Automatic Speech Recognition and Understanding Workshop, ASRU 2017 - Proceedings, vol. 2018-January, Institute of Electrical and Electronics Engineers Inc., pp. 671-676, 2017 IEEE Automatic Speech Recognition and Understanding Workshop, ASRU 2017, Okinawa, Japan, 17/12/16. https://doi.org/10.1109/ASRU.2017.8269001

Perceptual quality and modeling accuracy of excitation parameters in DLSTM-based speech synthesis systems. / Song, Eunwoo; Soong, Frank K.; Kang, Hong Goo.

2017 IEEE Automatic Speech Recognition and Understanding Workshop, ASRU 2017 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2018. p. 671-676 (2017 IEEE Automatic Speech Recognition and Understanding Workshop, ASRU 2017 - Proceedings; Vol. 2018-January).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Perceptual quality and modeling accuracy of excitation parameters in DLSTM-based speech synthesis systems

AU - Song, Eunwoo

AU - Soong, Frank K.

AU - Kang, Hong Goo

N1 - Funding Information: This research was supported by Microsoft Research and the MSIP (The Ministry of Science, ICT and Future Planning), Korea, under ICT/SW Creative research program supervised by the IITP (Institute for Information & Communications Technology Promotion). The authors would like to thank Feng-Long Xie, Microsoft Research Asia, Beijing, China, for conducting the listening tests. Publisher Copyright: © 2017 IEEE.

PY - 2018/1/24

Y1 - 2018/1/24

N2 - This paper investigates how the perceptual quality of the synthesized speech is affected by reconstruction errors in excitation signals generated by a deep learning-based statistical model. In this framework, the excitation signal obtained by an LPC inverse filter is first decomposed into harmonic and noise components using an improved time-frequency trajectory excitation (ITFTE) scheme, then they are trained and generated by a deep long short-term memory (DLSTM)-based speech synthesis system. By controlling the parametric dimension of the ITFTE vocoder, we analyze the impact of the harmonic and noise components to the perceptual quality of the synthesized speech. Both objective and subjective experimental results confirm that the maximum perceptually allowable spectral distortion for the harmonic spectrum of the generated excitation is ∼0.08 dB. On the other hand, the absolute spectral distortion in the noise components is meaningless, and only the spectral envelope is relevant to the perceptual quality.

AB - This paper investigates how the perceptual quality of the synthesized speech is affected by reconstruction errors in excitation signals generated by a deep learning-based statistical model. In this framework, the excitation signal obtained by an LPC inverse filter is first decomposed into harmonic and noise components using an improved time-frequency trajectory excitation (ITFTE) scheme, then they are trained and generated by a deep long short-term memory (DLSTM)-based speech synthesis system. By controlling the parametric dimension of the ITFTE vocoder, we analyze the impact of the harmonic and noise components to the perceptual quality of the synthesized speech. Both objective and subjective experimental results confirm that the maximum perceptually allowable spectral distortion for the harmonic spectrum of the generated excitation is ∼0.08 dB. On the other hand, the absolute spectral distortion in the noise components is meaningless, and only the spectral envelope is relevant to the perceptual quality.

UR - http://www.scopus.com/inward/record.url?scp=85050636605&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85050636605&partnerID=8YFLogxK

U2 - 10.1109/ASRU.2017.8269001

DO - 10.1109/ASRU.2017.8269001

M3 - Conference contribution

AN - SCOPUS:85050636605

T3 - 2017 IEEE Automatic Speech Recognition and Understanding Workshop, ASRU 2017 - Proceedings

SP - 671

EP - 676

BT - 2017 IEEE Automatic Speech Recognition and Understanding Workshop, ASRU 2017 - Proceedings

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2017 IEEE Automatic Speech Recognition and Understanding Workshop, ASRU 2017

Y2 - 16 December 2017 through 20 December 2017

ER -

Song E, Soong FK, Kang HG. Perceptual quality and modeling accuracy of excitation parameters in DLSTM-based speech synthesis systems. In 2017 IEEE Automatic Speech Recognition and Understanding Workshop, ASRU 2017 - Proceedings. Institute of Electrical and Electronics Engineers Inc. 2018. p. 671-676. (2017 IEEE Automatic Speech Recognition and Understanding Workshop, ASRU 2017 - Proceedings). doi: 10.1109/ASRU.2017.8269001

Perceptual quality and modeling accuracy of excitation parameters in DLSTM-based speech synthesis systems

Abstract

Publication series

Other

Bibliographical note

All Science Journal Classification (ASJC) codes

Access to Document

Other files and links

Fingerprint

Cite this