On integral value iteration for continuous-time linear systems

This paper investigates the properties of integral value iteration (I-VI) which is one of the reinforcement learning (RL) technique for solving online the continuous-time (CT) optimal control problems without using the system drift dynamics. The target I-VI is the one applied to CT linear quadratic regulation problems. As a result, two modes of global monotone convergence of I-VI are presented. One behaves like policy iteration (PI) (PI-mode of convergence) and the other is named VI-mode of convergence. All of the other properties - positive definiteness, stability, and relation between I-VI and integral PI - are presented within these two frameworks. Finally, numerical simulations are carried out to verify and further investigate these properties.