Effective lazy training method for deep Q-network in obstacle avoidance and path planning

Juan Wu, Seabyuk Shin, Cheong Gil Kim, Shin Dug Kim

Research output: Chapter in Book/Report/Conference proceedingConference contribution

4 Citations (Scopus)

Abstract

Deep reinforcement learning technique combines reinforcement learning and neural network for various applications. This paper is to propose an effective lazy training method for deep reinforcement learning, especially for deep Qnetwork combining neural network with Q-learning to be used for the obstacle avoidance and path planning applications. The proposed method can reduce the overall training time by designing a lazy learning method and a method removing unnecessary repetitions in the training step. These two methods can reduce a significant portion of total execution time without losing any required accuracy. The proposed method is evaluated for the obstacle avoidance and path planning tasks, where an agent trapped in an unknown environment is trying to find out the shortest path to the destination without any collision, through its self-study. And the experiment results show that the proposed method reduces 53.38% of training time on average, compared to the traditional method with no performance loss and make the training procedure more stable.

Original languageEnglish
Title of host publication2017 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2017
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1799-1804
Number of pages6
ISBN (Electronic)9781538616451
DOIs
Publication statusPublished - 2017 Nov 27
Event2017 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2017 - Banff, Canada
Duration: 2017 Oct 52017 Oct 8

Publication series

Name2017 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2017
Volume2017-January

Other

Other2017 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2017
CountryCanada
CityBanff
Period17/10/517/10/8

Fingerprint

Obstacle Avoidance
Path Planning
Collision avoidance
Motion planning
Reinforcement learning
Reinforcement Learning
Neural networks
Neural Networks
Q-learning
Training
Shortest path
Execution Time
Collision
Experiments
Unknown

All Science Journal Classification (ASJC) codes

  • Artificial Intelligence
  • Computer Science Applications
  • Human-Computer Interaction
  • Control and Optimization

Cite this

Wu, J., Shin, S., Kim, C. G., & Kim, S. D. (2017). Effective lazy training method for deep Q-network in obstacle avoidance and path planning. In 2017 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2017 (pp. 1799-1804). (2017 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2017; Vol. 2017-January). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/SMC.2017.8122877
Wu, Juan ; Shin, Seabyuk ; Kim, Cheong Gil ; Kim, Shin Dug. / Effective lazy training method for deep Q-network in obstacle avoidance and path planning. 2017 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2017. Institute of Electrical and Electronics Engineers Inc., 2017. pp. 1799-1804 (2017 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2017).
@inproceedings{5bee6c1311144530a2739b5a5579a0ed,
title = "Effective lazy training method for deep Q-network in obstacle avoidance and path planning",
abstract = "Deep reinforcement learning technique combines reinforcement learning and neural network for various applications. This paper is to propose an effective lazy training method for deep reinforcement learning, especially for deep Qnetwork combining neural network with Q-learning to be used for the obstacle avoidance and path planning applications. The proposed method can reduce the overall training time by designing a lazy learning method and a method removing unnecessary repetitions in the training step. These two methods can reduce a significant portion of total execution time without losing any required accuracy. The proposed method is evaluated for the obstacle avoidance and path planning tasks, where an agent trapped in an unknown environment is trying to find out the shortest path to the destination without any collision, through its self-study. And the experiment results show that the proposed method reduces 53.38{\%} of training time on average, compared to the traditional method with no performance loss and make the training procedure more stable.",
author = "Juan Wu and Seabyuk Shin and Kim, {Cheong Gil} and Kim, {Shin Dug}",
year = "2017",
month = "11",
day = "27",
doi = "10.1109/SMC.2017.8122877",
language = "English",
series = "2017 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2017",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "1799--1804",
booktitle = "2017 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2017",
address = "United States",

}

Wu, J, Shin, S, Kim, CG & Kim, SD 2017, Effective lazy training method for deep Q-network in obstacle avoidance and path planning. in 2017 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2017. 2017 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2017, vol. 2017-January, Institute of Electrical and Electronics Engineers Inc., pp. 1799-1804, 2017 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2017, Banff, Canada, 17/10/5. https://doi.org/10.1109/SMC.2017.8122877

Effective lazy training method for deep Q-network in obstacle avoidance and path planning. / Wu, Juan; Shin, Seabyuk; Kim, Cheong Gil; Kim, Shin Dug.

2017 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2017. Institute of Electrical and Electronics Engineers Inc., 2017. p. 1799-1804 (2017 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2017; Vol. 2017-January).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Effective lazy training method for deep Q-network in obstacle avoidance and path planning

AU - Wu, Juan

AU - Shin, Seabyuk

AU - Kim, Cheong Gil

AU - Kim, Shin Dug

PY - 2017/11/27

Y1 - 2017/11/27

N2 - Deep reinforcement learning technique combines reinforcement learning and neural network for various applications. This paper is to propose an effective lazy training method for deep reinforcement learning, especially for deep Qnetwork combining neural network with Q-learning to be used for the obstacle avoidance and path planning applications. The proposed method can reduce the overall training time by designing a lazy learning method and a method removing unnecessary repetitions in the training step. These two methods can reduce a significant portion of total execution time without losing any required accuracy. The proposed method is evaluated for the obstacle avoidance and path planning tasks, where an agent trapped in an unknown environment is trying to find out the shortest path to the destination without any collision, through its self-study. And the experiment results show that the proposed method reduces 53.38% of training time on average, compared to the traditional method with no performance loss and make the training procedure more stable.

AB - Deep reinforcement learning technique combines reinforcement learning and neural network for various applications. This paper is to propose an effective lazy training method for deep reinforcement learning, especially for deep Qnetwork combining neural network with Q-learning to be used for the obstacle avoidance and path planning applications. The proposed method can reduce the overall training time by designing a lazy learning method and a method removing unnecessary repetitions in the training step. These two methods can reduce a significant portion of total execution time without losing any required accuracy. The proposed method is evaluated for the obstacle avoidance and path planning tasks, where an agent trapped in an unknown environment is trying to find out the shortest path to the destination without any collision, through its self-study. And the experiment results show that the proposed method reduces 53.38% of training time on average, compared to the traditional method with no performance loss and make the training procedure more stable.

UR - http://www.scopus.com/inward/record.url?scp=85044107208&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85044107208&partnerID=8YFLogxK

U2 - 10.1109/SMC.2017.8122877

DO - 10.1109/SMC.2017.8122877

M3 - Conference contribution

AN - SCOPUS:85044107208

T3 - 2017 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2017

SP - 1799

EP - 1804

BT - 2017 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2017

PB - Institute of Electrical and Electronics Engineers Inc.

ER -

Wu J, Shin S, Kim CG, Kim SD. Effective lazy training method for deep Q-network in obstacle avoidance and path planning. In 2017 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2017. Institute of Electrical and Electronics Engineers Inc. 2017. p. 1799-1804. (2017 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2017). https://doi.org/10.1109/SMC.2017.8122877