Local Critic Training of Deep Neural Networks

Hojung Lee, Jong Seok Lee

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

This paper proposes a novel approach to train deep neural networks by unlocking the layer-wise dependency of backpropagation training. The approach employs additional modules called local critic networks besides the main network model to be trained, which are used to obtain error gradients without complete feedforward and backward propagation processes. We propose a cascaded learning strategy for these local networks. In addition, the approach is also useful from multi-model perspectives, including structural optimization of neural networks, computationally efficient progressive inference, and ensemble classification for performance improvement. Experimental results show the effectiveness of the proposed approach and suggest guidelines for determining appropriate algorithm parameters.

Original languageEnglish
Title of host publication2019 International Joint Conference on Neural Networks, IJCNN 2019
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781728119854
DOIs
Publication statusPublished - 2019 Jul
Event2019 International Joint Conference on Neural Networks, IJCNN 2019 - Budapest, Hungary
Duration: 2019 Jul 142019 Jul 19

Publication series

NameProceedings of the International Joint Conference on Neural Networks
Volume2019-July

Conference

Conference2019 International Joint Conference on Neural Networks, IJCNN 2019
CountryHungary
CityBudapest
Period19/7/1419/7/19

Fingerprint

Structural optimization
Backpropagation
Neural networks
Deep neural networks

All Science Journal Classification (ASJC) codes

  • Software
  • Artificial Intelligence

Cite this

Lee, H., & Lee, J. S. (2019). Local Critic Training of Deep Neural Networks. In 2019 International Joint Conference on Neural Networks, IJCNN 2019 [8851709] (Proceedings of the International Joint Conference on Neural Networks; Vol. 2019-July). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/IJCNN.2019.8851709
Lee, Hojung ; Lee, Jong Seok. / Local Critic Training of Deep Neural Networks. 2019 International Joint Conference on Neural Networks, IJCNN 2019. Institute of Electrical and Electronics Engineers Inc., 2019. (Proceedings of the International Joint Conference on Neural Networks).
@inproceedings{ae7433a65585469689e664347917150c,
title = "Local Critic Training of Deep Neural Networks",
abstract = "This paper proposes a novel approach to train deep neural networks by unlocking the layer-wise dependency of backpropagation training. The approach employs additional modules called local critic networks besides the main network model to be trained, which are used to obtain error gradients without complete feedforward and backward propagation processes. We propose a cascaded learning strategy for these local networks. In addition, the approach is also useful from multi-model perspectives, including structural optimization of neural networks, computationally efficient progressive inference, and ensemble classification for performance improvement. Experimental results show the effectiveness of the proposed approach and suggest guidelines for determining appropriate algorithm parameters.",
author = "Hojung Lee and Lee, {Jong Seok}",
year = "2019",
month = "7",
doi = "10.1109/IJCNN.2019.8851709",
language = "English",
series = "Proceedings of the International Joint Conference on Neural Networks",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
booktitle = "2019 International Joint Conference on Neural Networks, IJCNN 2019",
address = "United States",

}

Lee, H & Lee, JS 2019, Local Critic Training of Deep Neural Networks. in 2019 International Joint Conference on Neural Networks, IJCNN 2019., 8851709, Proceedings of the International Joint Conference on Neural Networks, vol. 2019-July, Institute of Electrical and Electronics Engineers Inc., 2019 International Joint Conference on Neural Networks, IJCNN 2019, Budapest, Hungary, 19/7/14. https://doi.org/10.1109/IJCNN.2019.8851709

Local Critic Training of Deep Neural Networks. / Lee, Hojung; Lee, Jong Seok.

2019 International Joint Conference on Neural Networks, IJCNN 2019. Institute of Electrical and Electronics Engineers Inc., 2019. 8851709 (Proceedings of the International Joint Conference on Neural Networks; Vol. 2019-July).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Local Critic Training of Deep Neural Networks

AU - Lee, Hojung

AU - Lee, Jong Seok

PY - 2019/7

Y1 - 2019/7

N2 - This paper proposes a novel approach to train deep neural networks by unlocking the layer-wise dependency of backpropagation training. The approach employs additional modules called local critic networks besides the main network model to be trained, which are used to obtain error gradients without complete feedforward and backward propagation processes. We propose a cascaded learning strategy for these local networks. In addition, the approach is also useful from multi-model perspectives, including structural optimization of neural networks, computationally efficient progressive inference, and ensemble classification for performance improvement. Experimental results show the effectiveness of the proposed approach and suggest guidelines for determining appropriate algorithm parameters.

AB - This paper proposes a novel approach to train deep neural networks by unlocking the layer-wise dependency of backpropagation training. The approach employs additional modules called local critic networks besides the main network model to be trained, which are used to obtain error gradients without complete feedforward and backward propagation processes. We propose a cascaded learning strategy for these local networks. In addition, the approach is also useful from multi-model perspectives, including structural optimization of neural networks, computationally efficient progressive inference, and ensemble classification for performance improvement. Experimental results show the effectiveness of the proposed approach and suggest guidelines for determining appropriate algorithm parameters.

UR - http://www.scopus.com/inward/record.url?scp=85073260731&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85073260731&partnerID=8YFLogxK

U2 - 10.1109/IJCNN.2019.8851709

DO - 10.1109/IJCNN.2019.8851709

M3 - Conference contribution

AN - SCOPUS:85073260731

T3 - Proceedings of the International Joint Conference on Neural Networks

BT - 2019 International Joint Conference on Neural Networks, IJCNN 2019

PB - Institute of Electrical and Electronics Engineers Inc.

ER -

Lee H, Lee JS. Local Critic Training of Deep Neural Networks. In 2019 International Joint Conference on Neural Networks, IJCNN 2019. Institute of Electrical and Electronics Engineers Inc. 2019. 8851709. (Proceedings of the International Joint Conference on Neural Networks). https://doi.org/10.1109/IJCNN.2019.8851709