Neural Network Reliability Enhancement Approach Using Dropout Underutilization in GPU

Dongsu Lee, Hyunyul Lim, Tae Hyun Kim, Sungho Kang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Recently, the researches on DNN (deep neural network) using GPUs has been actively conducted. The reason for using GPUs in DNN is that it reduces the learning time by using many computational cores. However, GPUs have no implements to support the reliable computing operations. It can exacerbate the reliability of the deep neural network. To ensure the reliability of the deep neural network, the proposed approach is to utilize the dropout technique used in MLP (Multi-Layer Perceptron) learning. In case of the dropout method, some threads in GPUs do not participate in the calculation and it causes GPUs underutilization. The proposed method uses the GPUs underutilization to support a reliable deep neural network. The proposed approach is available through using the idle neurons with adjacent calculating neurons in the dropout process. The experiment results show that the proposed approach is able to support the reliability issues in GPUs while executing deep neural network algorithms.

Original languageEnglish
Title of host publicationProceedings of TENCON 2018 - 2018 IEEE Region 10 Conference
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages2281-2286
Number of pages6
ISBN (Electronic)9781538654576
DOIs
Publication statusPublished - 2019 Feb 22
Event2018 IEEE Region 10 Conference, TENCON 2018 - Jeju, Korea, Republic of
Duration: 2018 Oct 282018 Oct 31

Publication series

NameIEEE Region 10 Annual International Conference, Proceedings/TENCON
Volume2018-October
ISSN (Print)2159-3442
ISSN (Electronic)2159-3450

Conference

Conference2018 IEEE Region 10 Conference, TENCON 2018
CountryKorea, Republic of
CityJeju
Period18/10/2818/10/31

Fingerprint

Neural networks
Neurons
Multilayer neural networks
Graphics processing unit
Deep neural networks
Experiments

All Science Journal Classification (ASJC) codes

  • Computer Science Applications
  • Electrical and Electronic Engineering

Cite this

Lee, D., Lim, H., Kim, T. H., & Kang, S. (2019). Neural Network Reliability Enhancement Approach Using Dropout Underutilization in GPU. In Proceedings of TENCON 2018 - 2018 IEEE Region 10 Conference (pp. 2281-2286). [8650184] (IEEE Region 10 Annual International Conference, Proceedings/TENCON; Vol. 2018-October). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/TENCON.2018.8650184
Lee, Dongsu ; Lim, Hyunyul ; Kim, Tae Hyun ; Kang, Sungho. / Neural Network Reliability Enhancement Approach Using Dropout Underutilization in GPU. Proceedings of TENCON 2018 - 2018 IEEE Region 10 Conference. Institute of Electrical and Electronics Engineers Inc., 2019. pp. 2281-2286 (IEEE Region 10 Annual International Conference, Proceedings/TENCON).
@inproceedings{ee2007d1737241dfa353175fd5db692c,
title = "Neural Network Reliability Enhancement Approach Using Dropout Underutilization in GPU",
abstract = "Recently, the researches on DNN (deep neural network) using GPUs has been actively conducted. The reason for using GPUs in DNN is that it reduces the learning time by using many computational cores. However, GPUs have no implements to support the reliable computing operations. It can exacerbate the reliability of the deep neural network. To ensure the reliability of the deep neural network, the proposed approach is to utilize the dropout technique used in MLP (Multi-Layer Perceptron) learning. In case of the dropout method, some threads in GPUs do not participate in the calculation and it causes GPUs underutilization. The proposed method uses the GPUs underutilization to support a reliable deep neural network. The proposed approach is available through using the idle neurons with adjacent calculating neurons in the dropout process. The experiment results show that the proposed approach is able to support the reliability issues in GPUs while executing deep neural network algorithms.",
author = "Dongsu Lee and Hyunyul Lim and Kim, {Tae Hyun} and Sungho Kang",
year = "2019",
month = "2",
day = "22",
doi = "10.1109/TENCON.2018.8650184",
language = "English",
series = "IEEE Region 10 Annual International Conference, Proceedings/TENCON",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "2281--2286",
booktitle = "Proceedings of TENCON 2018 - 2018 IEEE Region 10 Conference",
address = "United States",

}

Lee, D, Lim, H, Kim, TH & Kang, S 2019, Neural Network Reliability Enhancement Approach Using Dropout Underutilization in GPU. in Proceedings of TENCON 2018 - 2018 IEEE Region 10 Conference., 8650184, IEEE Region 10 Annual International Conference, Proceedings/TENCON, vol. 2018-October, Institute of Electrical and Electronics Engineers Inc., pp. 2281-2286, 2018 IEEE Region 10 Conference, TENCON 2018, Jeju, Korea, Republic of, 18/10/28. https://doi.org/10.1109/TENCON.2018.8650184

Neural Network Reliability Enhancement Approach Using Dropout Underutilization in GPU. / Lee, Dongsu; Lim, Hyunyul; Kim, Tae Hyun; Kang, Sungho.

Proceedings of TENCON 2018 - 2018 IEEE Region 10 Conference. Institute of Electrical and Electronics Engineers Inc., 2019. p. 2281-2286 8650184 (IEEE Region 10 Annual International Conference, Proceedings/TENCON; Vol. 2018-October).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Neural Network Reliability Enhancement Approach Using Dropout Underutilization in GPU

AU - Lee, Dongsu

AU - Lim, Hyunyul

AU - Kim, Tae Hyun

AU - Kang, Sungho

PY - 2019/2/22

Y1 - 2019/2/22

N2 - Recently, the researches on DNN (deep neural network) using GPUs has been actively conducted. The reason for using GPUs in DNN is that it reduces the learning time by using many computational cores. However, GPUs have no implements to support the reliable computing operations. It can exacerbate the reliability of the deep neural network. To ensure the reliability of the deep neural network, the proposed approach is to utilize the dropout technique used in MLP (Multi-Layer Perceptron) learning. In case of the dropout method, some threads in GPUs do not participate in the calculation and it causes GPUs underutilization. The proposed method uses the GPUs underutilization to support a reliable deep neural network. The proposed approach is available through using the idle neurons with adjacent calculating neurons in the dropout process. The experiment results show that the proposed approach is able to support the reliability issues in GPUs while executing deep neural network algorithms.

AB - Recently, the researches on DNN (deep neural network) using GPUs has been actively conducted. The reason for using GPUs in DNN is that it reduces the learning time by using many computational cores. However, GPUs have no implements to support the reliable computing operations. It can exacerbate the reliability of the deep neural network. To ensure the reliability of the deep neural network, the proposed approach is to utilize the dropout technique used in MLP (Multi-Layer Perceptron) learning. In case of the dropout method, some threads in GPUs do not participate in the calculation and it causes GPUs underutilization. The proposed method uses the GPUs underutilization to support a reliable deep neural network. The proposed approach is available through using the idle neurons with adjacent calculating neurons in the dropout process. The experiment results show that the proposed approach is able to support the reliability issues in GPUs while executing deep neural network algorithms.

UR - http://www.scopus.com/inward/record.url?scp=85063187356&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85063187356&partnerID=8YFLogxK

U2 - 10.1109/TENCON.2018.8650184

DO - 10.1109/TENCON.2018.8650184

M3 - Conference contribution

T3 - IEEE Region 10 Annual International Conference, Proceedings/TENCON

SP - 2281

EP - 2286

BT - Proceedings of TENCON 2018 - 2018 IEEE Region 10 Conference

PB - Institute of Electrical and Electronics Engineers Inc.

ER -

Lee D, Lim H, Kim TH, Kang S. Neural Network Reliability Enhancement Approach Using Dropout Underutilization in GPU. In Proceedings of TENCON 2018 - 2018 IEEE Region 10 Conference. Institute of Electrical and Electronics Engineers Inc. 2019. p. 2281-2286. 8650184. (IEEE Region 10 Annual International Conference, Proceedings/TENCON). https://doi.org/10.1109/TENCON.2018.8650184