Use of SAMC for Bayesian analysis of statistical models with intractable normalizing constants

Ick Hoon Jin, Faming Liang

Research output: Contribution to journalArticle

2 Citations (Scopus)

Abstract

Statistical inference for the models with intractable normalizing constants has attracted much attention. During the past two decades, various approximation- or simulation-based methods have been proposed for the problem, such as the Monte Carlo maximum likelihood method and the auxiliary variable Markov chain Monte Carlo methods. The Bayesian stochastic approximation Monte Carlo algorithm specifically addresses this problem: It works by sampling from a sequence of approximate distributions with their average converging to the target posterior distribution, where the approximate distributions can be achieved using the stochastic approximation Monte Carlo algorithm. A strong law of large numbers is established for the Bayesian stochastic approximation Monte Carlo estimator under mild conditions. Compared to the Monte Carlo maximum likelihood method, the Bayesian stochastic approximation Monte Carlo algorithm is more robust to the initial guess of model parameters. Compared to the auxiliary variable MCMC methods, the Bayesian stochastic approximation Monte Carlo algorithm avoids the requirement for perfect samples, and thus can be applied to many models for which perfect sampling is not available or very expensive. The Bayesian stochastic approximation Monte Carlo algorithm also provides a general framework for approximate Bayesian analysis.

Original languageEnglish
Pages (from-to)402-416
Number of pages15
JournalComputational Statistics and Data Analysis
Volume71
DOIs
Publication statusPublished - 2014 Jan 1

Fingerprint

Normalizing Constant
Stochastic Approximation
Bayesian Analysis
Monte Carlo Algorithm
Statistical Model
Approximation Algorithms
Auxiliary Variables
Maximum Likelihood Method
Maximum likelihood
Sampling
Perfect Sampling
MCMC Methods
Strong law of large numbers
Markov processes
Markov Chain Monte Carlo Methods
Guess
Statistical Inference
Posterior distribution
Monte Carlo methods
Statistical Models

All Science Journal Classification (ASJC) codes

  • Statistics and Probability
  • Computational Mathematics
  • Computational Theory and Mathematics
  • Applied Mathematics

Cite this

@article{1b45e8b35d254e7ebb30246615a8f8bf,
title = "Use of SAMC for Bayesian analysis of statistical models with intractable normalizing constants",
abstract = "Statistical inference for the models with intractable normalizing constants has attracted much attention. During the past two decades, various approximation- or simulation-based methods have been proposed for the problem, such as the Monte Carlo maximum likelihood method and the auxiliary variable Markov chain Monte Carlo methods. The Bayesian stochastic approximation Monte Carlo algorithm specifically addresses this problem: It works by sampling from a sequence of approximate distributions with their average converging to the target posterior distribution, where the approximate distributions can be achieved using the stochastic approximation Monte Carlo algorithm. A strong law of large numbers is established for the Bayesian stochastic approximation Monte Carlo estimator under mild conditions. Compared to the Monte Carlo maximum likelihood method, the Bayesian stochastic approximation Monte Carlo algorithm is more robust to the initial guess of model parameters. Compared to the auxiliary variable MCMC methods, the Bayesian stochastic approximation Monte Carlo algorithm avoids the requirement for perfect samples, and thus can be applied to many models for which perfect sampling is not available or very expensive. The Bayesian stochastic approximation Monte Carlo algorithm also provides a general framework for approximate Bayesian analysis.",
author = "Jin, {Ick Hoon} and Faming Liang",
year = "2014",
month = "1",
day = "1",
doi = "10.1016/j.csda.2012.07.005",
language = "English",
volume = "71",
pages = "402--416",
journal = "Computational Statistics and Data Analysis",
issn = "0167-9473",
publisher = "Elsevier",

}

Use of SAMC for Bayesian analysis of statistical models with intractable normalizing constants. / Jin, Ick Hoon; Liang, Faming.

In: Computational Statistics and Data Analysis, Vol. 71, 01.01.2014, p. 402-416.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Use of SAMC for Bayesian analysis of statistical models with intractable normalizing constants

AU - Jin, Ick Hoon

AU - Liang, Faming

PY - 2014/1/1

Y1 - 2014/1/1

N2 - Statistical inference for the models with intractable normalizing constants has attracted much attention. During the past two decades, various approximation- or simulation-based methods have been proposed for the problem, such as the Monte Carlo maximum likelihood method and the auxiliary variable Markov chain Monte Carlo methods. The Bayesian stochastic approximation Monte Carlo algorithm specifically addresses this problem: It works by sampling from a sequence of approximate distributions with their average converging to the target posterior distribution, where the approximate distributions can be achieved using the stochastic approximation Monte Carlo algorithm. A strong law of large numbers is established for the Bayesian stochastic approximation Monte Carlo estimator under mild conditions. Compared to the Monte Carlo maximum likelihood method, the Bayesian stochastic approximation Monte Carlo algorithm is more robust to the initial guess of model parameters. Compared to the auxiliary variable MCMC methods, the Bayesian stochastic approximation Monte Carlo algorithm avoids the requirement for perfect samples, and thus can be applied to many models for which perfect sampling is not available or very expensive. The Bayesian stochastic approximation Monte Carlo algorithm also provides a general framework for approximate Bayesian analysis.

AB - Statistical inference for the models with intractable normalizing constants has attracted much attention. During the past two decades, various approximation- or simulation-based methods have been proposed for the problem, such as the Monte Carlo maximum likelihood method and the auxiliary variable Markov chain Monte Carlo methods. The Bayesian stochastic approximation Monte Carlo algorithm specifically addresses this problem: It works by sampling from a sequence of approximate distributions with their average converging to the target posterior distribution, where the approximate distributions can be achieved using the stochastic approximation Monte Carlo algorithm. A strong law of large numbers is established for the Bayesian stochastic approximation Monte Carlo estimator under mild conditions. Compared to the Monte Carlo maximum likelihood method, the Bayesian stochastic approximation Monte Carlo algorithm is more robust to the initial guess of model parameters. Compared to the auxiliary variable MCMC methods, the Bayesian stochastic approximation Monte Carlo algorithm avoids the requirement for perfect samples, and thus can be applied to many models for which perfect sampling is not available or very expensive. The Bayesian stochastic approximation Monte Carlo algorithm also provides a general framework for approximate Bayesian analysis.

UR - http://www.scopus.com/inward/record.url?scp=84889086254&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84889086254&partnerID=8YFLogxK

U2 - 10.1016/j.csda.2012.07.005

DO - 10.1016/j.csda.2012.07.005

M3 - Article

AN - SCOPUS:84889086254

VL - 71

SP - 402

EP - 416

JO - Computational Statistics and Data Analysis

JF - Computational Statistics and Data Analysis

SN - 0167-9473

ER -