Comparisons of continuous-time and discrete-time Q-learning schemes for adaptive linear quadratic control

Tae Yoon Chun, Jae Young Lee, Jin Bae Park, Yoon Ho Choi

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

In this paper, we compare two online model-free Q-learning schemes for adaptive linear quadratic (LQ) control of discrete-time (DT) and continuous-time (CT) dynamical systems. Both Q-learning schemes come from the optimality principles, but the DT and CT g-learning is designed with different β-functions. This difference may results in the different exploration properties and convergence speeds. Numerical simulations with an ideal DC motor are carried out to further investigate and compare the Q-learning methods.

Original languageEnglish
Title of host publication2012 Proceedings of SICE Annual Conference, SICE 2012
PublisherSociety of Instrument and Control Engineers (SICE)
Pages1228-1233
Number of pages6
ISBN (Print)9781467322591
Publication statusPublished - 2012 Jan 1
Event2012 51st Annual Conference on of the Society of Instrument and Control Engineers of Japan, SICE 2012 - Akita, Japan
Duration: 2012 Aug 202012 Aug 23

Other

Other2012 51st Annual Conference on of the Society of Instrument and Control Engineers of Japan, SICE 2012
CountryJapan
CityAkita
Period12/8/2012/8/23

Fingerprint

DC motors
Dynamical systems
Computer simulation

All Science Journal Classification (ASJC) codes

  • Control and Systems Engineering
  • Computer Science Applications
  • Electrical and Electronic Engineering

Cite this

Chun, T. Y., Lee, J. Y., Park, J. B., & Choi, Y. H. (2012). Comparisons of continuous-time and discrete-time Q-learning schemes for adaptive linear quadratic control. In 2012 Proceedings of SICE Annual Conference, SICE 2012 (pp. 1228-1233). [6318633] Society of Instrument and Control Engineers (SICE).
Chun, Tae Yoon ; Lee, Jae Young ; Park, Jin Bae ; Choi, Yoon Ho. / Comparisons of continuous-time and discrete-time Q-learning schemes for adaptive linear quadratic control. 2012 Proceedings of SICE Annual Conference, SICE 2012. Society of Instrument and Control Engineers (SICE), 2012. pp. 1228-1233
@inproceedings{9e485e803f92466d8131847b22e8e946,
title = "Comparisons of continuous-time and discrete-time Q-learning schemes for adaptive linear quadratic control",
abstract = "In this paper, we compare two online model-free Q-learning schemes for adaptive linear quadratic (LQ) control of discrete-time (DT) and continuous-time (CT) dynamical systems. Both Q-learning schemes come from the optimality principles, but the DT and CT g-learning is designed with different β-functions. This difference may results in the different exploration properties and convergence speeds. Numerical simulations with an ideal DC motor are carried out to further investigate and compare the Q-learning methods.",
author = "Chun, {Tae Yoon} and Lee, {Jae Young} and Park, {Jin Bae} and Choi, {Yoon Ho}",
year = "2012",
month = "1",
day = "1",
language = "English",
isbn = "9781467322591",
pages = "1228--1233",
booktitle = "2012 Proceedings of SICE Annual Conference, SICE 2012",
publisher = "Society of Instrument and Control Engineers (SICE)",

}

Chun, TY, Lee, JY, Park, JB & Choi, YH 2012, Comparisons of continuous-time and discrete-time Q-learning schemes for adaptive linear quadratic control. in 2012 Proceedings of SICE Annual Conference, SICE 2012., 6318633, Society of Instrument and Control Engineers (SICE), pp. 1228-1233, 2012 51st Annual Conference on of the Society of Instrument and Control Engineers of Japan, SICE 2012, Akita, Japan, 12/8/20.

Comparisons of continuous-time and discrete-time Q-learning schemes for adaptive linear quadratic control. / Chun, Tae Yoon; Lee, Jae Young; Park, Jin Bae; Choi, Yoon Ho.

2012 Proceedings of SICE Annual Conference, SICE 2012. Society of Instrument and Control Engineers (SICE), 2012. p. 1228-1233 6318633.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Comparisons of continuous-time and discrete-time Q-learning schemes for adaptive linear quadratic control

AU - Chun, Tae Yoon

AU - Lee, Jae Young

AU - Park, Jin Bae

AU - Choi, Yoon Ho

PY - 2012/1/1

Y1 - 2012/1/1

N2 - In this paper, we compare two online model-free Q-learning schemes for adaptive linear quadratic (LQ) control of discrete-time (DT) and continuous-time (CT) dynamical systems. Both Q-learning schemes come from the optimality principles, but the DT and CT g-learning is designed with different β-functions. This difference may results in the different exploration properties and convergence speeds. Numerical simulations with an ideal DC motor are carried out to further investigate and compare the Q-learning methods.

AB - In this paper, we compare two online model-free Q-learning schemes for adaptive linear quadratic (LQ) control of discrete-time (DT) and continuous-time (CT) dynamical systems. Both Q-learning schemes come from the optimality principles, but the DT and CT g-learning is designed with different β-functions. This difference may results in the different exploration properties and convergence speeds. Numerical simulations with an ideal DC motor are carried out to further investigate and compare the Q-learning methods.

UR - http://www.scopus.com/inward/record.url?scp=84869412855&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84869412855&partnerID=8YFLogxK

M3 - Conference contribution

SN - 9781467322591

SP - 1228

EP - 1233

BT - 2012 Proceedings of SICE Annual Conference, SICE 2012

PB - Society of Instrument and Control Engineers (SICE)

ER -

Chun TY, Lee JY, Park JB, Choi YH. Comparisons of continuous-time and discrete-time Q-learning schemes for adaptive linear quadratic control. In 2012 Proceedings of SICE Annual Conference, SICE 2012. Society of Instrument and Control Engineers (SICE). 2012. p. 1228-1233. 6318633