Adaptive dynamic programming for discrete-time linear quadratic regulation based on multirate generalised policy iteration

Tae Yoon Chun, Jae Young Lee, Jin Bae Park, Yoon Ho Choi

Research output: Contribution to journalArticlepeer-review

3 Citations (Scopus)

Abstract

In this paper, we propose two multirate generalised policy iteration (GPI) algorithms applied to discrete-time linear quadratic regulation problems. The proposed algorithms are extensions of the existing GPI algorithm that consists of the approximate policy evaluation and policy improvement steps. The two proposed schemes, named heuristic dynamic programming (HDP) and dual HDP (DHP), based on multirate GPI, use multi-step estimation (M-step Bellman equation) at the approximate policy evaluation step for estimating the value function and its gradient called costate, respectively. Then, we show that these two methods with the same update horizon can be considered equivalent in the iteration domain. Furthermore, monotonically increasing and decreasing convergences, so called value iteration (VI)-mode and policy iteration (PI)-mode convergences, are proved to hold for the proposed multirate GPIs. Further, general convergence properties in terms of eigenvalues are also studied. The data-driven online implementation methods for the proposed HDP and DHP are demonstrated and finally, we present the results of numerical simulations performed to verify the effectiveness of the proposed methods.

Original languageEnglish
Pages (from-to)1223-1240
Number of pages18
JournalInternational Journal of Control
Volume91
Issue number6
DOIs
Publication statusPublished - 2018 Jun 3

Bibliographical note

Funding Information:
The authors appreciate the associate editor and anonymous reviewers for their valuable suggestions. No potential conflict of interest was reported by the authors.

All Science Journal Classification (ASJC) codes

  • Control and Systems Engineering
  • Computer Science Applications

Fingerprint Dive into the research topics of 'Adaptive dynamic programming for discrete-time linear quadratic regulation based on multirate generalised policy iteration'. Together they form a unique fingerprint.

Cite this