Stability and monotone convergence of generalised policy iteration for discrete-Time linear quadratic regulations

Tae Yoon Chun, Jae Young Lee, Jin Bae Park, Yoon Ho Choi

Research output: Contribution to journalArticle

2 Citations (Scopus)

Abstract

In this paper, we analyse the convergence and stability properties of generalised policy iteration (GPI) applied to discrete-Time linear quadratic regulation problems. GPI is one kind of the generalised adaptive dynamic programming methods used for solving optimal control problems, and is composed of policy evaluation and policy improvement steps. To analyse the convergence and stability of GPI, the dynamic programming (DP) operator is defined. Then, GPI and its equivalent formulas are presented based on the notation of DP operator. The convergence of the approximate value function to the exact one in policy evaluation is proven based on the equivalent formulas. Furthermore, the positive semi-definiteness, stability, and the monotone convergence (PI-mode and VI-mode convergence) of GPI are presented under certain conditions on the initial value function. The online least square method is also presented for the implementation of GPI. Finally, some numerical simulations are carried out to verify the effectiveness of GPI as well as to further investigate the convergence and stability properties.

Original languageEnglish
Pages (from-to)437-450
Number of pages14
JournalInternational Journal of Control
Volume89
Issue number3
DOIs
Publication statusPublished - 2016 Mar 3

Fingerprint

Dynamic programming
Mathematical operators
Computer simulation

All Science Journal Classification (ASJC) codes

  • Control and Systems Engineering
  • Computer Science Applications

Cite this

@article{ec8175fe47784add8bfa3b6d2bd3ec3c,
title = "Stability and monotone convergence of generalised policy iteration for discrete-Time linear quadratic regulations",
abstract = "In this paper, we analyse the convergence and stability properties of generalised policy iteration (GPI) applied to discrete-Time linear quadratic regulation problems. GPI is one kind of the generalised adaptive dynamic programming methods used for solving optimal control problems, and is composed of policy evaluation and policy improvement steps. To analyse the convergence and stability of GPI, the dynamic programming (DP) operator is defined. Then, GPI and its equivalent formulas are presented based on the notation of DP operator. The convergence of the approximate value function to the exact one in policy evaluation is proven based on the equivalent formulas. Furthermore, the positive semi-definiteness, stability, and the monotone convergence (PI-mode and VI-mode convergence) of GPI are presented under certain conditions on the initial value function. The online least square method is also presented for the implementation of GPI. Finally, some numerical simulations are carried out to verify the effectiveness of GPI as well as to further investigate the convergence and stability properties.",
author = "Chun, {Tae Yoon} and Lee, {Jae Young} and Park, {Jin Bae} and Choi, {Yoon Ho}",
year = "2016",
month = "3",
day = "3",
doi = "10.1080/00207179.2015.1079737",
language = "English",
volume = "89",
pages = "437--450",
journal = "International Journal of Control",
issn = "0020-7179",
publisher = "Taylor and Francis Ltd.",
number = "3",

}

Stability and monotone convergence of generalised policy iteration for discrete-Time linear quadratic regulations. / Chun, Tae Yoon; Lee, Jae Young; Park, Jin Bae; Choi, Yoon Ho.

In: International Journal of Control, Vol. 89, No. 3, 03.03.2016, p. 437-450.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Stability and monotone convergence of generalised policy iteration for discrete-Time linear quadratic regulations

AU - Chun, Tae Yoon

AU - Lee, Jae Young

AU - Park, Jin Bae

AU - Choi, Yoon Ho

PY - 2016/3/3

Y1 - 2016/3/3

N2 - In this paper, we analyse the convergence and stability properties of generalised policy iteration (GPI) applied to discrete-Time linear quadratic regulation problems. GPI is one kind of the generalised adaptive dynamic programming methods used for solving optimal control problems, and is composed of policy evaluation and policy improvement steps. To analyse the convergence and stability of GPI, the dynamic programming (DP) operator is defined. Then, GPI and its equivalent formulas are presented based on the notation of DP operator. The convergence of the approximate value function to the exact one in policy evaluation is proven based on the equivalent formulas. Furthermore, the positive semi-definiteness, stability, and the monotone convergence (PI-mode and VI-mode convergence) of GPI are presented under certain conditions on the initial value function. The online least square method is also presented for the implementation of GPI. Finally, some numerical simulations are carried out to verify the effectiveness of GPI as well as to further investigate the convergence and stability properties.

AB - In this paper, we analyse the convergence and stability properties of generalised policy iteration (GPI) applied to discrete-Time linear quadratic regulation problems. GPI is one kind of the generalised adaptive dynamic programming methods used for solving optimal control problems, and is composed of policy evaluation and policy improvement steps. To analyse the convergence and stability of GPI, the dynamic programming (DP) operator is defined. Then, GPI and its equivalent formulas are presented based on the notation of DP operator. The convergence of the approximate value function to the exact one in policy evaluation is proven based on the equivalent formulas. Furthermore, the positive semi-definiteness, stability, and the monotone convergence (PI-mode and VI-mode convergence) of GPI are presented under certain conditions on the initial value function. The online least square method is also presented for the implementation of GPI. Finally, some numerical simulations are carried out to verify the effectiveness of GPI as well as to further investigate the convergence and stability properties.

UR - http://www.scopus.com/inward/record.url?scp=84955410397&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84955410397&partnerID=8YFLogxK

U2 - 10.1080/00207179.2015.1079737

DO - 10.1080/00207179.2015.1079737

M3 - Article

VL - 89

SP - 437

EP - 450

JO - International Journal of Control

JF - International Journal of Control

SN - 0020-7179

IS - 3

ER -