Model-free approximate dynamic programming for continuous-time linear systems

Jae Young Lee, Jin Bae Park, Yoon Ho Choi

Research output: Chapter in Book/Report/Conference proceedingConference contribution

4 Citations (Scopus)

Abstract

In this paper, a novel online approximate dynamic programming (ADP) technique for completely unknown continuous-time linear systems is proposed to solve the infinite horizon linear quadratic (LQ) optimal control problems. For relaxing the assumption of the known input coupling matrix, the conventional LQ optimal control problem is converted into the proposed cheap control problem. Then, the ADP agent iteratively solves this cheap optimal control problem in online fashion to obtain the near-optimal solution of the conventional LQ optimal control problem. In addition, we mathematically prove the approximation property of the cheap optimal control problem with respect to the conventional LQ optimal control problem. The numerical simulation for ideal DC motor shows the applicability of the proposed ADP algorithm.

Original languageEnglish
Title of host publicationProceedings of the 48th IEEE Conference on Decision and Control held jointly with 2009 28th Chinese Control Conference, CDC/CCC 2009
Pages5009-5014
Number of pages6
DOIs
Publication statusPublished - 2009 Dec 1
Event48th IEEE Conference on Decision and Control held jointly with 2009 28th Chinese Control Conference, CDC/CCC 2009 - Shanghai, China
Duration: 2009 Dec 152009 Dec 18

Other

Other48th IEEE Conference on Decision and Control held jointly with 2009 28th Chinese Control Conference, CDC/CCC 2009
CountryChina
CityShanghai
Period09/12/1509/12/18

Fingerprint

Approximate Dynamic Programming
Continuous-time Systems
Dynamic programming
Linear systems
Optimal Control Problem
Linear Systems
Model
DC Motor
Infinite Horizon
Approximation Property
DC motors
Control Problem
Optimal Solution
Unknown
Numerical Simulation
Computer simulation

All Science Journal Classification (ASJC) codes

  • Control and Systems Engineering
  • Modelling and Simulation
  • Control and Optimization

Cite this

Lee, J. Y., Park, J. B., & Choi, Y. H. (2009). Model-free approximate dynamic programming for continuous-time linear systems. In Proceedings of the 48th IEEE Conference on Decision and Control held jointly with 2009 28th Chinese Control Conference, CDC/CCC 2009 (pp. 5009-5014). [5400371] https://doi.org/10.1109/CDC.2009.5400371
Lee, Jae Young ; Park, Jin Bae ; Choi, Yoon Ho. / Model-free approximate dynamic programming for continuous-time linear systems. Proceedings of the 48th IEEE Conference on Decision and Control held jointly with 2009 28th Chinese Control Conference, CDC/CCC 2009. 2009. pp. 5009-5014
@inproceedings{703e7eeac3b84162a11fc7fb036dd9d1,
title = "Model-free approximate dynamic programming for continuous-time linear systems",
abstract = "In this paper, a novel online approximate dynamic programming (ADP) technique for completely unknown continuous-time linear systems is proposed to solve the infinite horizon linear quadratic (LQ) optimal control problems. For relaxing the assumption of the known input coupling matrix, the conventional LQ optimal control problem is converted into the proposed cheap control problem. Then, the ADP agent iteratively solves this cheap optimal control problem in online fashion to obtain the near-optimal solution of the conventional LQ optimal control problem. In addition, we mathematically prove the approximation property of the cheap optimal control problem with respect to the conventional LQ optimal control problem. The numerical simulation for ideal DC motor shows the applicability of the proposed ADP algorithm.",
author = "Lee, {Jae Young} and Park, {Jin Bae} and Choi, {Yoon Ho}",
year = "2009",
month = "12",
day = "1",
doi = "10.1109/CDC.2009.5400371",
language = "English",
isbn = "9781424438716",
pages = "5009--5014",
booktitle = "Proceedings of the 48th IEEE Conference on Decision and Control held jointly with 2009 28th Chinese Control Conference, CDC/CCC 2009",

}

Lee, JY, Park, JB & Choi, YH 2009, Model-free approximate dynamic programming for continuous-time linear systems. in Proceedings of the 48th IEEE Conference on Decision and Control held jointly with 2009 28th Chinese Control Conference, CDC/CCC 2009., 5400371, pp. 5009-5014, 48th IEEE Conference on Decision and Control held jointly with 2009 28th Chinese Control Conference, CDC/CCC 2009, Shanghai, China, 09/12/15. https://doi.org/10.1109/CDC.2009.5400371

Model-free approximate dynamic programming for continuous-time linear systems. / Lee, Jae Young; Park, Jin Bae; Choi, Yoon Ho.

Proceedings of the 48th IEEE Conference on Decision and Control held jointly with 2009 28th Chinese Control Conference, CDC/CCC 2009. 2009. p. 5009-5014 5400371.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Model-free approximate dynamic programming for continuous-time linear systems

AU - Lee, Jae Young

AU - Park, Jin Bae

AU - Choi, Yoon Ho

PY - 2009/12/1

Y1 - 2009/12/1

N2 - In this paper, a novel online approximate dynamic programming (ADP) technique for completely unknown continuous-time linear systems is proposed to solve the infinite horizon linear quadratic (LQ) optimal control problems. For relaxing the assumption of the known input coupling matrix, the conventional LQ optimal control problem is converted into the proposed cheap control problem. Then, the ADP agent iteratively solves this cheap optimal control problem in online fashion to obtain the near-optimal solution of the conventional LQ optimal control problem. In addition, we mathematically prove the approximation property of the cheap optimal control problem with respect to the conventional LQ optimal control problem. The numerical simulation for ideal DC motor shows the applicability of the proposed ADP algorithm.

AB - In this paper, a novel online approximate dynamic programming (ADP) technique for completely unknown continuous-time linear systems is proposed to solve the infinite horizon linear quadratic (LQ) optimal control problems. For relaxing the assumption of the known input coupling matrix, the conventional LQ optimal control problem is converted into the proposed cheap control problem. Then, the ADP agent iteratively solves this cheap optimal control problem in online fashion to obtain the near-optimal solution of the conventional LQ optimal control problem. In addition, we mathematically prove the approximation property of the cheap optimal control problem with respect to the conventional LQ optimal control problem. The numerical simulation for ideal DC motor shows the applicability of the proposed ADP algorithm.

UR - http://www.scopus.com/inward/record.url?scp=77950824225&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=77950824225&partnerID=8YFLogxK

U2 - 10.1109/CDC.2009.5400371

DO - 10.1109/CDC.2009.5400371

M3 - Conference contribution

AN - SCOPUS:77950824225

SN - 9781424438716

SP - 5009

EP - 5014

BT - Proceedings of the 48th IEEE Conference on Decision and Control held jointly with 2009 28th Chinese Control Conference, CDC/CCC 2009

ER -

Lee JY, Park JB, Choi YH. Model-free approximate dynamic programming for continuous-time linear systems. In Proceedings of the 48th IEEE Conference on Decision and Control held jointly with 2009 28th Chinese Control Conference, CDC/CCC 2009. 2009. p. 5009-5014. 5400371 https://doi.org/10.1109/CDC.2009.5400371