On integral value iteration for continuous-time linear systems

Jae Young Lee; Jin Bae Park; Yoon Ho Choi

On integral value iteration for continuous-time linear systems

Jae Young Lee, Jin Bae Park, Yoon Ho Choi

Department of Electrical and Electronic Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

2 Citations (Scopus)

Abstract

This paper investigates the properties of integral value iteration (I-VI) which is one of the reinforcement learning (RL) technique for solving online the continuous-time (CT) optimal control problems without using the system drift dynamics. The target I-VI is the one applied to CT linear quadratic regulation problems. As a result, two modes of global monotone convergence of I-VI are presented. One behaves like policy iteration (PI) (PI-mode of convergence) and the other is named VI-mode of convergence. All of the other properties - positive definiteness, stability, and relation between I-VI and integral PI - are presented within these two frameworks. Finally, numerical simulations are carried out to verify and further investigate these properties.

Original language	English
Title of host publication	2013 American Control Conference, ACC 2013
Pages	4215-4220
Number of pages	6
Publication status	Published - 2013 Sep 11
Event	2013 1st American Control Conference, ACC 2013 - Washington, DC, United States Duration: 2013 Jun 17 → 2013 Jun 19

Other

Other	2013 1st American Control Conference, ACC 2013
Country	United States
City	Washington, DC
Period	13/6/17 → 13/6/19

All Science Journal Classification (ASJC) codes

Electrical and Electronic Engineering

Access to Document

Fingerprint Dive into the research topics of 'On integral value iteration for continuous-time linear systems'. Together they form a unique fingerprint.

Cite this

Lee, J. Y., Park, J. B., & Choi, Y. H. (2013). On integral value iteration for continuous-time linear systems. In 2013 American Control Conference, ACC 2013 (pp. 4215-4220). [6580487]

@inproceedings{ed3346677a484b979abf9f6c77aa5506,

title = "On integral value iteration for continuous-time linear systems",

abstract = "This paper investigates the properties of integral value iteration (I-VI) which is one of the reinforcement learning (RL) technique for solving online the continuous-time (CT) optimal control problems without using the system drift dynamics. The target I-VI is the one applied to CT linear quadratic regulation problems. As a result, two modes of global monotone convergence of I-VI are presented. One behaves like policy iteration (PI) (PI-mode of convergence) and the other is named VI-mode of convergence. All of the other properties - positive definiteness, stability, and relation between I-VI and integral PI - are presented within these two frameworks. Finally, numerical simulations are carried out to verify and further investigate these properties.",

author = "Lee, {Jae Young} and Park, {Jin Bae} and Choi, {Yoon Ho}",

year = "2013",

month = sep,

day = "11",

language = "English",

isbn = "9781479901777",

pages = "4215--4220",

booktitle = "2013 American Control Conference, ACC 2013",

note = "2013 1st American Control Conference, ACC 2013 ; Conference date: 17-06-2013 Through 19-06-2013",

}

TY - GEN

T1 - On integral value iteration for continuous-time linear systems

AU - Lee, Jae Young

AU - Park, Jin Bae

AU - Choi, Yoon Ho

PY - 2013/9/11

Y1 - 2013/9/11

N2 - This paper investigates the properties of integral value iteration (I-VI) which is one of the reinforcement learning (RL) technique for solving online the continuous-time (CT) optimal control problems without using the system drift dynamics. The target I-VI is the one applied to CT linear quadratic regulation problems. As a result, two modes of global monotone convergence of I-VI are presented. One behaves like policy iteration (PI) (PI-mode of convergence) and the other is named VI-mode of convergence. All of the other properties - positive definiteness, stability, and relation between I-VI and integral PI - are presented within these two frameworks. Finally, numerical simulations are carried out to verify and further investigate these properties.

AB - This paper investigates the properties of integral value iteration (I-VI) which is one of the reinforcement learning (RL) technique for solving online the continuous-time (CT) optimal control problems without using the system drift dynamics. The target I-VI is the one applied to CT linear quadratic regulation problems. As a result, two modes of global monotone convergence of I-VI are presented. One behaves like policy iteration (PI) (PI-mode of convergence) and the other is named VI-mode of convergence. All of the other properties - positive definiteness, stability, and relation between I-VI and integral PI - are presented within these two frameworks. Finally, numerical simulations are carried out to verify and further investigate these properties.

UR - http://www.scopus.com/inward/record.url?scp=84883494157&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84883494157&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:84883494157

SN - 9781479901777

SP - 4215

EP - 4220

BT - 2013 American Control Conference, ACC 2013

T2 - 2013 1st American Control Conference, ACC 2013

Y2 - 17 June 2013 through 19 June 2013

ER -