End-to-End Steering Controller with CNN-based Closed-loop Feedback for Autonomous Vehicles

Junekyo Jhung, Il Bae, Jaeyoung Moon, Taewoo Kim, Jincheol Kim, Shiho Kim

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

Many significant research achievements in the past few decades have demonstrated that convolutional neural networks (CNNs) could be capable of steering wheel control which is the basic and essential maneuver of autonomous vehicles. We propose an end-to-end steering controller with CNN-based closed-loop feedback for autonomous vehicles that improves driving performance compared to traditional CNN-based approaches. This paper demonstrates that the proposed neural network, DAVE-2SKY, is able to learn to inference steering wheel angles for the lateral control of self-driving vehicles through initial supervised pre-training and subsequent reinforced closed-loop post-training with images from a camera mounted on the vehicle. We used the PreScan simulator and Caffe deep learning framework for training under diversified circumstances in a software in the loop (SIL) simulation environment. We used DRIVE TM PX2 computer to implement a self-driving car for an experimental validation of the proposed end-to-end controller. The performance of the proposed system has been investigated under simulations and on-road tests as well. This work shows that the CNN-based end-to-end controller performs robust steering control even under partially observable road conditions, which indicates the possibility of fully self-driving vehicles controlled by CNN-based end-to-end steering controllers.

Original languageEnglish
Title of host publication2018 IEEE Intelligent Vehicles Symposium, IV 2018
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages617-622
Number of pages6
ISBN (Electronic)9781538644522
DOIs
Publication statusPublished - 2018 Oct 18
Event2018 IEEE Intelligent Vehicles Symposium, IV 2018 - Changshu, Suzhou, China
Duration: 2018 Sep 262018 Sep 30

Publication series

NameIEEE Intelligent Vehicles Symposium, Proceedings
Volume2018-June

Other

Other2018 IEEE Intelligent Vehicles Symposium, IV 2018
CountryChina
CityChangshu, Suzhou
Period18/9/2618/9/30

Fingerprint

Autonomous Vehicles
Closed-loop
Neural Networks
Neural networks
Feedback
Controller
Controllers
Wheel
Wheels
Experimental Validation
Simulation Environment
Lateral
Simulator
Railroad cars
Simulators
Camera
Cameras
Angle
Software
Demonstrate

All Science Journal Classification (ASJC) codes

  • Computer Science Applications
  • Automotive Engineering
  • Modelling and Simulation

Cite this

Jhung, J., Bae, I., Moon, J., Kim, T., Kim, J., & Kim, S. (2018). End-to-End Steering Controller with CNN-based Closed-loop Feedback for Autonomous Vehicles. In 2018 IEEE Intelligent Vehicles Symposium, IV 2018 (pp. 617-622). [8500440] (IEEE Intelligent Vehicles Symposium, Proceedings; Vol. 2018-June). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/IVS.2018.8500440
Jhung, Junekyo ; Bae, Il ; Moon, Jaeyoung ; Kim, Taewoo ; Kim, Jincheol ; Kim, Shiho. / End-to-End Steering Controller with CNN-based Closed-loop Feedback for Autonomous Vehicles. 2018 IEEE Intelligent Vehicles Symposium, IV 2018. Institute of Electrical and Electronics Engineers Inc., 2018. pp. 617-622 (IEEE Intelligent Vehicles Symposium, Proceedings).
@inproceedings{4455a0c041b1421885baac4d6aae2a1b,
title = "End-to-End Steering Controller with CNN-based Closed-loop Feedback for Autonomous Vehicles",
abstract = "Many significant research achievements in the past few decades have demonstrated that convolutional neural networks (CNNs) could be capable of steering wheel control which is the basic and essential maneuver of autonomous vehicles. We propose an end-to-end steering controller with CNN-based closed-loop feedback for autonomous vehicles that improves driving performance compared to traditional CNN-based approaches. This paper demonstrates that the proposed neural network, DAVE-2SKY, is able to learn to inference steering wheel angles for the lateral control of self-driving vehicles through initial supervised pre-training and subsequent reinforced closed-loop post-training with images from a camera mounted on the vehicle. We used the PreScan simulator and Caffe deep learning framework for training under diversified circumstances in a software in the loop (SIL) simulation environment. We used DRIVE TM PX2 computer to implement a self-driving car for an experimental validation of the proposed end-to-end controller. The performance of the proposed system has been investigated under simulations and on-road tests as well. This work shows that the CNN-based end-to-end controller performs robust steering control even under partially observable road conditions, which indicates the possibility of fully self-driving vehicles controlled by CNN-based end-to-end steering controllers.",
author = "Junekyo Jhung and Il Bae and Jaeyoung Moon and Taewoo Kim and Jincheol Kim and Shiho Kim",
year = "2018",
month = "10",
day = "18",
doi = "10.1109/IVS.2018.8500440",
language = "English",
series = "IEEE Intelligent Vehicles Symposium, Proceedings",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "617--622",
booktitle = "2018 IEEE Intelligent Vehicles Symposium, IV 2018",
address = "United States",

}

Jhung, J, Bae, I, Moon, J, Kim, T, Kim, J & Kim, S 2018, End-to-End Steering Controller with CNN-based Closed-loop Feedback for Autonomous Vehicles. in 2018 IEEE Intelligent Vehicles Symposium, IV 2018., 8500440, IEEE Intelligent Vehicles Symposium, Proceedings, vol. 2018-June, Institute of Electrical and Electronics Engineers Inc., pp. 617-622, 2018 IEEE Intelligent Vehicles Symposium, IV 2018, Changshu, Suzhou, China, 18/9/26. https://doi.org/10.1109/IVS.2018.8500440

End-to-End Steering Controller with CNN-based Closed-loop Feedback for Autonomous Vehicles. / Jhung, Junekyo; Bae, Il; Moon, Jaeyoung; Kim, Taewoo; Kim, Jincheol; Kim, Shiho.

2018 IEEE Intelligent Vehicles Symposium, IV 2018. Institute of Electrical and Electronics Engineers Inc., 2018. p. 617-622 8500440 (IEEE Intelligent Vehicles Symposium, Proceedings; Vol. 2018-June).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - End-to-End Steering Controller with CNN-based Closed-loop Feedback for Autonomous Vehicles

AU - Jhung, Junekyo

AU - Bae, Il

AU - Moon, Jaeyoung

AU - Kim, Taewoo

AU - Kim, Jincheol

AU - Kim, Shiho

PY - 2018/10/18

Y1 - 2018/10/18

N2 - Many significant research achievements in the past few decades have demonstrated that convolutional neural networks (CNNs) could be capable of steering wheel control which is the basic and essential maneuver of autonomous vehicles. We propose an end-to-end steering controller with CNN-based closed-loop feedback for autonomous vehicles that improves driving performance compared to traditional CNN-based approaches. This paper demonstrates that the proposed neural network, DAVE-2SKY, is able to learn to inference steering wheel angles for the lateral control of self-driving vehicles through initial supervised pre-training and subsequent reinforced closed-loop post-training with images from a camera mounted on the vehicle. We used the PreScan simulator and Caffe deep learning framework for training under diversified circumstances in a software in the loop (SIL) simulation environment. We used DRIVE TM PX2 computer to implement a self-driving car for an experimental validation of the proposed end-to-end controller. The performance of the proposed system has been investigated under simulations and on-road tests as well. This work shows that the CNN-based end-to-end controller performs robust steering control even under partially observable road conditions, which indicates the possibility of fully self-driving vehicles controlled by CNN-based end-to-end steering controllers.

AB - Many significant research achievements in the past few decades have demonstrated that convolutional neural networks (CNNs) could be capable of steering wheel control which is the basic and essential maneuver of autonomous vehicles. We propose an end-to-end steering controller with CNN-based closed-loop feedback for autonomous vehicles that improves driving performance compared to traditional CNN-based approaches. This paper demonstrates that the proposed neural network, DAVE-2SKY, is able to learn to inference steering wheel angles for the lateral control of self-driving vehicles through initial supervised pre-training and subsequent reinforced closed-loop post-training with images from a camera mounted on the vehicle. We used the PreScan simulator and Caffe deep learning framework for training under diversified circumstances in a software in the loop (SIL) simulation environment. We used DRIVE TM PX2 computer to implement a self-driving car for an experimental validation of the proposed end-to-end controller. The performance of the proposed system has been investigated under simulations and on-road tests as well. This work shows that the CNN-based end-to-end controller performs robust steering control even under partially observable road conditions, which indicates the possibility of fully self-driving vehicles controlled by CNN-based end-to-end steering controllers.

UR - http://www.scopus.com/inward/record.url?scp=85056787482&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85056787482&partnerID=8YFLogxK

U2 - 10.1109/IVS.2018.8500440

DO - 10.1109/IVS.2018.8500440

M3 - Conference contribution

T3 - IEEE Intelligent Vehicles Symposium, Proceedings

SP - 617

EP - 622

BT - 2018 IEEE Intelligent Vehicles Symposium, IV 2018

PB - Institute of Electrical and Electronics Engineers Inc.

ER -

Jhung J, Bae I, Moon J, Kim T, Kim J, Kim S. End-to-End Steering Controller with CNN-based Closed-loop Feedback for Autonomous Vehicles. In 2018 IEEE Intelligent Vehicles Symposium, IV 2018. Institute of Electrical and Electronics Engineers Inc. 2018. p. 617-622. 8500440. (IEEE Intelligent Vehicles Symposium, Proceedings). https://doi.org/10.1109/IVS.2018.8500440