Online sparse gaussian process regression and its applications

Ananth Ranganathan, Ming Hsuan Yang, Jeffrey Ho

Research output: Contribution to journalArticle

71 Citations (Scopus)

Abstract

We present a new Gaussian process (GP) inference algorithm, called online sparse matrix Gaussian processes (OSMGP), and demonstrate its merits by applying it to the problems of head pose estimation and visual tracking. The OSMGP is based upon the observation that for kernels with local support, the Gram matrix is typically sparse. Maintaining and updating the sparse Cholesky factor of the Gram matrix can be done efficiently using Givens rotations. This leads to an exact, online algorithm whose update time scales linearly with the size of the Gram matrix. Further, we provide a method for constant time operation of the OSMGP using matrix downdates. The downdates maintain the Cholesky factor at a constant size by removing certain rows and columns corresponding to discarded training examples. We demonstrate that, using these matrix downdates, online hyperparameter estimation can be included at cost linear in the number of total training examples. We describe a robust appearance-based head pose estimation system based upon the OSMGP. Numerous experiments and comparisons with existing methods using a large dataset system demonstrate the efficiency and accuracy of our system. Further, to showcase the applicability of OSMGP to a wide variety of problems, we also describe a regression-based visual tracking method. Experiments show that our OSMGP algorithm generalizes well using online learning.

Original languageEnglish
Article number5549909
Pages (from-to)391-404
Number of pages14
JournalIEEE Transactions on Image Processing
Volume20
Issue number2
DOIs
Publication statusPublished - 2011 Feb 1

Fingerprint

Experiments
Costs

All Science Journal Classification (ASJC) codes

  • Software
  • Computer Graphics and Computer-Aided Design

Cite this

Ranganathan, Ananth ; Yang, Ming Hsuan ; Ho, Jeffrey. / Online sparse gaussian process regression and its applications. In: IEEE Transactions on Image Processing. 2011 ; Vol. 20, No. 2. pp. 391-404.
@article{5def977ce2eb4c60a8c23b40c976f8b3,
title = "Online sparse gaussian process regression and its applications",
abstract = "We present a new Gaussian process (GP) inference algorithm, called online sparse matrix Gaussian processes (OSMGP), and demonstrate its merits by applying it to the problems of head pose estimation and visual tracking. The OSMGP is based upon the observation that for kernels with local support, the Gram matrix is typically sparse. Maintaining and updating the sparse Cholesky factor of the Gram matrix can be done efficiently using Givens rotations. This leads to an exact, online algorithm whose update time scales linearly with the size of the Gram matrix. Further, we provide a method for constant time operation of the OSMGP using matrix downdates. The downdates maintain the Cholesky factor at a constant size by removing certain rows and columns corresponding to discarded training examples. We demonstrate that, using these matrix downdates, online hyperparameter estimation can be included at cost linear in the number of total training examples. We describe a robust appearance-based head pose estimation system based upon the OSMGP. Numerous experiments and comparisons with existing methods using a large dataset system demonstrate the efficiency and accuracy of our system. Further, to showcase the applicability of OSMGP to a wide variety of problems, we also describe a regression-based visual tracking method. Experiments show that our OSMGP algorithm generalizes well using online learning.",
author = "Ananth Ranganathan and Yang, {Ming Hsuan} and Jeffrey Ho",
year = "2011",
month = "2",
day = "1",
doi = "10.1109/TIP.2010.2066984",
language = "English",
volume = "20",
pages = "391--404",
journal = "IEEE Transactions on Image Processing",
issn = "1057-7149",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
number = "2",

}

Online sparse gaussian process regression and its applications. / Ranganathan, Ananth; Yang, Ming Hsuan; Ho, Jeffrey.

In: IEEE Transactions on Image Processing, Vol. 20, No. 2, 5549909, 01.02.2011, p. 391-404.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Online sparse gaussian process regression and its applications

AU - Ranganathan, Ananth

AU - Yang, Ming Hsuan

AU - Ho, Jeffrey

PY - 2011/2/1

Y1 - 2011/2/1

N2 - We present a new Gaussian process (GP) inference algorithm, called online sparse matrix Gaussian processes (OSMGP), and demonstrate its merits by applying it to the problems of head pose estimation and visual tracking. The OSMGP is based upon the observation that for kernels with local support, the Gram matrix is typically sparse. Maintaining and updating the sparse Cholesky factor of the Gram matrix can be done efficiently using Givens rotations. This leads to an exact, online algorithm whose update time scales linearly with the size of the Gram matrix. Further, we provide a method for constant time operation of the OSMGP using matrix downdates. The downdates maintain the Cholesky factor at a constant size by removing certain rows and columns corresponding to discarded training examples. We demonstrate that, using these matrix downdates, online hyperparameter estimation can be included at cost linear in the number of total training examples. We describe a robust appearance-based head pose estimation system based upon the OSMGP. Numerous experiments and comparisons with existing methods using a large dataset system demonstrate the efficiency and accuracy of our system. Further, to showcase the applicability of OSMGP to a wide variety of problems, we also describe a regression-based visual tracking method. Experiments show that our OSMGP algorithm generalizes well using online learning.

AB - We present a new Gaussian process (GP) inference algorithm, called online sparse matrix Gaussian processes (OSMGP), and demonstrate its merits by applying it to the problems of head pose estimation and visual tracking. The OSMGP is based upon the observation that for kernels with local support, the Gram matrix is typically sparse. Maintaining and updating the sparse Cholesky factor of the Gram matrix can be done efficiently using Givens rotations. This leads to an exact, online algorithm whose update time scales linearly with the size of the Gram matrix. Further, we provide a method for constant time operation of the OSMGP using matrix downdates. The downdates maintain the Cholesky factor at a constant size by removing certain rows and columns corresponding to discarded training examples. We demonstrate that, using these matrix downdates, online hyperparameter estimation can be included at cost linear in the number of total training examples. We describe a robust appearance-based head pose estimation system based upon the OSMGP. Numerous experiments and comparisons with existing methods using a large dataset system demonstrate the efficiency and accuracy of our system. Further, to showcase the applicability of OSMGP to a wide variety of problems, we also describe a regression-based visual tracking method. Experiments show that our OSMGP algorithm generalizes well using online learning.

UR - http://www.scopus.com/inward/record.url?scp=79551662807&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=79551662807&partnerID=8YFLogxK

U2 - 10.1109/TIP.2010.2066984

DO - 10.1109/TIP.2010.2066984

M3 - Article

C2 - 20716500

AN - SCOPUS:79551662807

VL - 20

SP - 391

EP - 404

JO - IEEE Transactions on Image Processing

JF - IEEE Transactions on Image Processing

SN - 1057-7149

IS - 2

M1 - 5549909

ER -