On generalized policy iteration for continuous-time linear systems

Jae Young Lee, Tae Yoon Chun, Jin Bae Park, Yoon Ho Choi

Research output: Chapter in Book/Report/Conference proceedingConference contribution

5 Citations (Scopus)

Abstract

This paper investigate the mathematical properties of generalized policy iteration (GPI) applied to a class of continuous-time linear systems with unknown internal dynamics. GPI is a class of dynamic programming (DP) method to solve an optimal control problem by using two consecutive steps-policy evaluation and policy improvement. We first provide several formula equivalent to GPI, and as a result, reveal its relations to linear quadratic optimal control problems and the fact that the computational complexity due to backup operations in policy evaluation steps can be lessened by increasing the time horizon of GPI. A variety of local stability and convergence criteria is also provided with the connection to the convergence speed. Finally, several numerical simulations are performed to verify the results.

Original languageEnglish
Title of host publication2011 50th IEEE Conference on Decision and Control and European Control Conference, CDC-ECC 2011
Pages1722-1728
Number of pages7
DOIs
Publication statusPublished - 2011 Dec 1
Event2011 50th IEEE Conference on Decision and Control and European Control Conference, CDC-ECC 2011 - Orlando, FL, United States
Duration: 2011 Dec 122011 Dec 15

Other

Other2011 50th IEEE Conference on Decision and Control and European Control Conference, CDC-ECC 2011
CountryUnited States
CityOrlando, FL
Period11/12/1211/12/15

Fingerprint

Policy Iteration
Continuous-time Systems
Linear systems
Linear Systems
Dynamic programming
Optimal Control Problem
Computational complexity
Convergence Criteria
Local Convergence
Computer simulation
Convergence Speed
Evaluation
Local Stability
Stability and Convergence
Stability Criteria
Dynamic Programming
Consecutive
Horizon
Computational Complexity
Verify

All Science Journal Classification (ASJC) codes

  • Control and Systems Engineering
  • Modelling and Simulation
  • Control and Optimization

Cite this

Lee, J. Y., Chun, T. Y., Park, J. B., & Choi, Y. H. (2011). On generalized policy iteration for continuous-time linear systems. In 2011 50th IEEE Conference on Decision and Control and European Control Conference, CDC-ECC 2011 (pp. 1722-1728). [6161462] https://doi.org/10.1109/CDC.2011.6161462
Lee, Jae Young ; Chun, Tae Yoon ; Park, Jin Bae ; Choi, Yoon Ho. / On generalized policy iteration for continuous-time linear systems. 2011 50th IEEE Conference on Decision and Control and European Control Conference, CDC-ECC 2011. 2011. pp. 1722-1728
@inproceedings{15a0d9f7f3be48e8bcb89d737c70f2a5,
title = "On generalized policy iteration for continuous-time linear systems",
abstract = "This paper investigate the mathematical properties of generalized policy iteration (GPI) applied to a class of continuous-time linear systems with unknown internal dynamics. GPI is a class of dynamic programming (DP) method to solve an optimal control problem by using two consecutive steps-policy evaluation and policy improvement. We first provide several formula equivalent to GPI, and as a result, reveal its relations to linear quadratic optimal control problems and the fact that the computational complexity due to backup operations in policy evaluation steps can be lessened by increasing the time horizon of GPI. A variety of local stability and convergence criteria is also provided with the connection to the convergence speed. Finally, several numerical simulations are performed to verify the results.",
author = "Lee, {Jae Young} and Chun, {Tae Yoon} and Park, {Jin Bae} and Choi, {Yoon Ho}",
year = "2011",
month = "12",
day = "1",
doi = "10.1109/CDC.2011.6161462",
language = "English",
isbn = "9781612848006",
pages = "1722--1728",
booktitle = "2011 50th IEEE Conference on Decision and Control and European Control Conference, CDC-ECC 2011",

}

Lee, JY, Chun, TY, Park, JB & Choi, YH 2011, On generalized policy iteration for continuous-time linear systems. in 2011 50th IEEE Conference on Decision and Control and European Control Conference, CDC-ECC 2011., 6161462, pp. 1722-1728, 2011 50th IEEE Conference on Decision and Control and European Control Conference, CDC-ECC 2011, Orlando, FL, United States, 11/12/12. https://doi.org/10.1109/CDC.2011.6161462

On generalized policy iteration for continuous-time linear systems. / Lee, Jae Young; Chun, Tae Yoon; Park, Jin Bae; Choi, Yoon Ho.

2011 50th IEEE Conference on Decision and Control and European Control Conference, CDC-ECC 2011. 2011. p. 1722-1728 6161462.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - On generalized policy iteration for continuous-time linear systems

AU - Lee, Jae Young

AU - Chun, Tae Yoon

AU - Park, Jin Bae

AU - Choi, Yoon Ho

PY - 2011/12/1

Y1 - 2011/12/1

N2 - This paper investigate the mathematical properties of generalized policy iteration (GPI) applied to a class of continuous-time linear systems with unknown internal dynamics. GPI is a class of dynamic programming (DP) method to solve an optimal control problem by using two consecutive steps-policy evaluation and policy improvement. We first provide several formula equivalent to GPI, and as a result, reveal its relations to linear quadratic optimal control problems and the fact that the computational complexity due to backup operations in policy evaluation steps can be lessened by increasing the time horizon of GPI. A variety of local stability and convergence criteria is also provided with the connection to the convergence speed. Finally, several numerical simulations are performed to verify the results.

AB - This paper investigate the mathematical properties of generalized policy iteration (GPI) applied to a class of continuous-time linear systems with unknown internal dynamics. GPI is a class of dynamic programming (DP) method to solve an optimal control problem by using two consecutive steps-policy evaluation and policy improvement. We first provide several formula equivalent to GPI, and as a result, reveal its relations to linear quadratic optimal control problems and the fact that the computational complexity due to backup operations in policy evaluation steps can be lessened by increasing the time horizon of GPI. A variety of local stability and convergence criteria is also provided with the connection to the convergence speed. Finally, several numerical simulations are performed to verify the results.

UR - http://www.scopus.com/inward/record.url?scp=84860677254&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84860677254&partnerID=8YFLogxK

U2 - 10.1109/CDC.2011.6161462

DO - 10.1109/CDC.2011.6161462

M3 - Conference contribution

SN - 9781612848006

SP - 1722

EP - 1728

BT - 2011 50th IEEE Conference on Decision and Control and European Control Conference, CDC-ECC 2011

ER -

Lee JY, Chun TY, Park JB, Choi YH. On generalized policy iteration for continuous-time linear systems. In 2011 50th IEEE Conference on Decision and Control and European Control Conference, CDC-ECC 2011. 2011. p. 1722-1728. 6161462 https://doi.org/10.1109/CDC.2011.6161462