Model-free approximate dynamic programming for continuous-time linear systems

Jae Young Lee; Jin Bae Park; Yoon Ho Choi

doi:10.1109/CDC.2009.5400371

Model-free approximate dynamic programming for continuous-time linear systems

Jae Young Lee, Jin Bae Park, Yoon Ho Choi

Department of Electrical and Electronic Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

5 Citations (Scopus)

Abstract

In this paper, a novel online approximate dynamic programming (ADP) technique for completely unknown continuous-time linear systems is proposed to solve the infinite horizon linear quadratic (LQ) optimal control problems. For relaxing the assumption of the known input coupling matrix, the conventional LQ optimal control problem is converted into the proposed cheap control problem. Then, the ADP agent iteratively solves this cheap optimal control problem in online fashion to obtain the near-optimal solution of the conventional LQ optimal control problem. In addition, we mathematically prove the approximation property of the cheap optimal control problem with respect to the conventional LQ optimal control problem. The numerical simulation for ideal DC motor shows the applicability of the proposed ADP algorithm.

Original language	English
Title of host publication	Proceedings of the 48th IEEE Conference on Decision and Control held jointly with 2009 28th Chinese Control Conference, CDC/CCC 2009
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	5009-5014
Number of pages	6
ISBN (Print)	9781424438716
DOIs	https://doi.org/10.1109/CDC.2009.5400371
Publication status	Published - 2009
Event	48th IEEE Conference on Decision and Control held jointly with 2009 28th Chinese Control Conference, CDC/CCC 2009 - Shanghai, China Duration: 2009 Dec 15 → 2009 Dec 18

Publication series

Name	Proceedings of the IEEE Conference on Decision and Control
ISSN (Print)	0743-1546
ISSN (Electronic)	2576-2370

Other

Other	48th IEEE Conference on Decision and Control held jointly with 2009 28th Chinese Control Conference, CDC/CCC 2009
Country/Territory	China
City	Shanghai
Period	09/12/15 → 09/12/18

All Science Journal Classification (ASJC) codes

Control and Systems Engineering
Modelling and Simulation
Control and Optimization

Access to Document

10.1109/CDC.2009.5400371

Cite this

Lee, J. Y., Park, J. B., & Choi, Y. H. (2009). Model-free approximate dynamic programming for continuous-time linear systems. In Proceedings of the 48th IEEE Conference on Decision and Control held jointly with 2009 28th Chinese Control Conference, CDC/CCC 2009 (pp. 5009-5014). [5400371] (Proceedings of the IEEE Conference on Decision and Control). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/CDC.2009.5400371

Lee, Jae Young ; Park, Jin Bae ; Choi, Yoon Ho. / Model-free approximate dynamic programming for continuous-time linear systems. Proceedings of the 48th IEEE Conference on Decision and Control held jointly with 2009 28th Chinese Control Conference, CDC/CCC 2009. Institute of Electrical and Electronics Engineers Inc., 2009. pp. 5009-5014 (Proceedings of the IEEE Conference on Decision and Control).

@inproceedings{703e7eeac3b84162a11fc7fb036dd9d1,

title = "Model-free approximate dynamic programming for continuous-time linear systems",

abstract = "In this paper, a novel online approximate dynamic programming (ADP) technique for completely unknown continuous-time linear systems is proposed to solve the infinite horizon linear quadratic (LQ) optimal control problems. For relaxing the assumption of the known input coupling matrix, the conventional LQ optimal control problem is converted into the proposed cheap control problem. Then, the ADP agent iteratively solves this cheap optimal control problem in online fashion to obtain the near-optimal solution of the conventional LQ optimal control problem. In addition, we mathematically prove the approximation property of the cheap optimal control problem with respect to the conventional LQ optimal control problem. The numerical simulation for ideal DC motor shows the applicability of the proposed ADP algorithm.",

author = "Lee, {Jae Young} and Park, {Jin Bae} and Choi, {Yoon Ho}",

year = "2009",

doi = "10.1109/CDC.2009.5400371",

language = "English",

isbn = "9781424438716",

series = "Proceedings of the IEEE Conference on Decision and Control",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "5009--5014",

booktitle = "Proceedings of the 48th IEEE Conference on Decision and Control held jointly with 2009 28th Chinese Control Conference, CDC/CCC 2009",

address = "United States",

note = "48th IEEE Conference on Decision and Control held jointly with 2009 28th Chinese Control Conference, CDC/CCC 2009 ; Conference date: 15-12-2009 Through 18-12-2009",

}

Lee, JY, Park, JB & Choi, YH 2009, Model-free approximate dynamic programming for continuous-time linear systems. in Proceedings of the 48th IEEE Conference on Decision and Control held jointly with 2009 28th Chinese Control Conference, CDC/CCC 2009., 5400371, Proceedings of the IEEE Conference on Decision and Control, Institute of Electrical and Electronics Engineers Inc., pp. 5009-5014, 48th IEEE Conference on Decision and Control held jointly with 2009 28th Chinese Control Conference, CDC/CCC 2009, Shanghai, China, 09/12/15. https://doi.org/10.1109/CDC.2009.5400371

Model-free approximate dynamic programming for continuous-time linear systems. / Lee, Jae Young; Park, Jin Bae; Choi, Yoon Ho.

Proceedings of the 48th IEEE Conference on Decision and Control held jointly with 2009 28th Chinese Control Conference, CDC/CCC 2009. Institute of Electrical and Electronics Engineers Inc., 2009. p. 5009-5014 5400371 (Proceedings of the IEEE Conference on Decision and Control).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Model-free approximate dynamic programming for continuous-time linear systems

AU - Lee, Jae Young

AU - Park, Jin Bae

AU - Choi, Yoon Ho

PY - 2009

Y1 - 2009

N2 - In this paper, a novel online approximate dynamic programming (ADP) technique for completely unknown continuous-time linear systems is proposed to solve the infinite horizon linear quadratic (LQ) optimal control problems. For relaxing the assumption of the known input coupling matrix, the conventional LQ optimal control problem is converted into the proposed cheap control problem. Then, the ADP agent iteratively solves this cheap optimal control problem in online fashion to obtain the near-optimal solution of the conventional LQ optimal control problem. In addition, we mathematically prove the approximation property of the cheap optimal control problem with respect to the conventional LQ optimal control problem. The numerical simulation for ideal DC motor shows the applicability of the proposed ADP algorithm.

AB - In this paper, a novel online approximate dynamic programming (ADP) technique for completely unknown continuous-time linear systems is proposed to solve the infinite horizon linear quadratic (LQ) optimal control problems. For relaxing the assumption of the known input coupling matrix, the conventional LQ optimal control problem is converted into the proposed cheap control problem. Then, the ADP agent iteratively solves this cheap optimal control problem in online fashion to obtain the near-optimal solution of the conventional LQ optimal control problem. In addition, we mathematically prove the approximation property of the cheap optimal control problem with respect to the conventional LQ optimal control problem. The numerical simulation for ideal DC motor shows the applicability of the proposed ADP algorithm.

UR - http://www.scopus.com/inward/record.url?scp=77950824225&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=77950824225&partnerID=8YFLogxK

U2 - 10.1109/CDC.2009.5400371

DO - 10.1109/CDC.2009.5400371

M3 - Conference contribution

AN - SCOPUS:77950824225

SN - 9781424438716

T3 - Proceedings of the IEEE Conference on Decision and Control

SP - 5009

EP - 5014

BT - Proceedings of the 48th IEEE Conference on Decision and Control held jointly with 2009 28th Chinese Control Conference, CDC/CCC 2009

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 48th IEEE Conference on Decision and Control held jointly with 2009 28th Chinese Control Conference, CDC/CCC 2009

Y2 - 15 December 2009 through 18 December 2009

ER -

Lee JY, Park JB, Choi YH. Model-free approximate dynamic programming for continuous-time linear systems. In Proceedings of the 48th IEEE Conference on Decision and Control held jointly with 2009 28th Chinese Control Conference, CDC/CCC 2009. Institute of Electrical and Electronics Engineers Inc. 2009. p. 5009-5014. 5400371. (Proceedings of the IEEE Conference on Decision and Control). doi: 10.1109/CDC.2009.5400371

Model-free approximate dynamic programming for continuous-time linear systems

Abstract

Publication series

Other

All Science Journal Classification (ASJC) codes

Access to Document

Other files and links

Fingerprint

Cite this