Theory and Applications of Hybrid Simulated Annealing

Jong Seok Lee, Cheol Hoon Park, Touradj Ebrahimi

Research output: Chapter in Book/Report/Conference proceedingChapter

4 Citations (Scopus)

Abstract

Local optimization techniques such as gradient-based methods and the expectation-maximization algorithm have an advantage of fast convergence but do not guarantee convergence to the global optimum. On the other hand, global optimization techniques based on stochastic approaches such as evolutionary algorithms and simulated annealing provide the possibility of global convergence, which is accomplished at the expense of computational and time complexity. This chapter aims at demonstrating how these two approaches can be effectively combined for improved convergence speed and quality of the solution. In particular, a hybrid method, called hybrid simulated annealing (HSA), is presented, where a simulated annealing algorithm is combined with local optimization methods. First, its general procedure and mathematical convergence properties are described. Then, its two example applications are presented, namely, optimization of hidden Markov models for visual speech recognition and optimization of radial basis function networks for pattern classification, in order to show how the HSA algorithm can be successfully adopted for solving real-world problems effectively. As an appendix, the source code for multi-dimensional Cauchy random number generation is provided, which is essential for implementation of the presented method.

Original languageEnglish
Title of host publicationHandbook of Optimization
Subtitle of host publicationFrom Classical to Modern Approach
EditorsIvan Zelinka, Vaclav Snasel, Ajith Abraham
Pages395-422
Number of pages28
DOIs
Publication statusPublished - 2013 Oct 18

Publication series

NameIntelligent Systems Reference Library
Volume38
ISSN (Print)1868-4394
ISSN (Electronic)1868-4408

Fingerprint

Simulated annealing
Random number generation
Radial basis function networks
Global optimization
Hidden Markov models
Speech recognition
Evolutionary algorithms
Pattern recognition
guarantee
Optimization techniques
Simulated annealing algorithm

All Science Journal Classification (ASJC) codes

  • Computer Science(all)
  • Information Systems and Management
  • Library and Information Sciences

Cite this

Lee, J. S., Park, C. H., & Ebrahimi, T. (2013). Theory and Applications of Hybrid Simulated Annealing. In I. Zelinka, V. Snasel, & A. Abraham (Eds.), Handbook of Optimization: From Classical to Modern Approach (pp. 395-422). (Intelligent Systems Reference Library; Vol. 38). https://doi.org/10.1007/978-3-642-30504-7_16
Lee, Jong Seok ; Park, Cheol Hoon ; Ebrahimi, Touradj. / Theory and Applications of Hybrid Simulated Annealing. Handbook of Optimization: From Classical to Modern Approach. editor / Ivan Zelinka ; Vaclav Snasel ; Ajith Abraham. 2013. pp. 395-422 (Intelligent Systems Reference Library).
@inbook{7a84bfb82c0145469774a0349058a8d0,
title = "Theory and Applications of Hybrid Simulated Annealing",
abstract = "Local optimization techniques such as gradient-based methods and the expectation-maximization algorithm have an advantage of fast convergence but do not guarantee convergence to the global optimum. On the other hand, global optimization techniques based on stochastic approaches such as evolutionary algorithms and simulated annealing provide the possibility of global convergence, which is accomplished at the expense of computational and time complexity. This chapter aims at demonstrating how these two approaches can be effectively combined for improved convergence speed and quality of the solution. In particular, a hybrid method, called hybrid simulated annealing (HSA), is presented, where a simulated annealing algorithm is combined with local optimization methods. First, its general procedure and mathematical convergence properties are described. Then, its two example applications are presented, namely, optimization of hidden Markov models for visual speech recognition and optimization of radial basis function networks for pattern classification, in order to show how the HSA algorithm can be successfully adopted for solving real-world problems effectively. As an appendix, the source code for multi-dimensional Cauchy random number generation is provided, which is essential for implementation of the presented method.",
author = "Lee, {Jong Seok} and Park, {Cheol Hoon} and Touradj Ebrahimi",
year = "2013",
month = "10",
day = "18",
doi = "10.1007/978-3-642-30504-7_16",
language = "English",
isbn = "9783642305030",
series = "Intelligent Systems Reference Library",
pages = "395--422",
editor = "Ivan Zelinka and Vaclav Snasel and Ajith Abraham",
booktitle = "Handbook of Optimization",

}

Lee, JS, Park, CH & Ebrahimi, T 2013, Theory and Applications of Hybrid Simulated Annealing. in I Zelinka, V Snasel & A Abraham (eds), Handbook of Optimization: From Classical to Modern Approach. Intelligent Systems Reference Library, vol. 38, pp. 395-422. https://doi.org/10.1007/978-3-642-30504-7_16

Theory and Applications of Hybrid Simulated Annealing. / Lee, Jong Seok; Park, Cheol Hoon; Ebrahimi, Touradj.

Handbook of Optimization: From Classical to Modern Approach. ed. / Ivan Zelinka; Vaclav Snasel; Ajith Abraham. 2013. p. 395-422 (Intelligent Systems Reference Library; Vol. 38).

Research output: Chapter in Book/Report/Conference proceedingChapter

TY - CHAP

T1 - Theory and Applications of Hybrid Simulated Annealing

AU - Lee, Jong Seok

AU - Park, Cheol Hoon

AU - Ebrahimi, Touradj

PY - 2013/10/18

Y1 - 2013/10/18

N2 - Local optimization techniques such as gradient-based methods and the expectation-maximization algorithm have an advantage of fast convergence but do not guarantee convergence to the global optimum. On the other hand, global optimization techniques based on stochastic approaches such as evolutionary algorithms and simulated annealing provide the possibility of global convergence, which is accomplished at the expense of computational and time complexity. This chapter aims at demonstrating how these two approaches can be effectively combined for improved convergence speed and quality of the solution. In particular, a hybrid method, called hybrid simulated annealing (HSA), is presented, where a simulated annealing algorithm is combined with local optimization methods. First, its general procedure and mathematical convergence properties are described. Then, its two example applications are presented, namely, optimization of hidden Markov models for visual speech recognition and optimization of radial basis function networks for pattern classification, in order to show how the HSA algorithm can be successfully adopted for solving real-world problems effectively. As an appendix, the source code for multi-dimensional Cauchy random number generation is provided, which is essential for implementation of the presented method.

AB - Local optimization techniques such as gradient-based methods and the expectation-maximization algorithm have an advantage of fast convergence but do not guarantee convergence to the global optimum. On the other hand, global optimization techniques based on stochastic approaches such as evolutionary algorithms and simulated annealing provide the possibility of global convergence, which is accomplished at the expense of computational and time complexity. This chapter aims at demonstrating how these two approaches can be effectively combined for improved convergence speed and quality of the solution. In particular, a hybrid method, called hybrid simulated annealing (HSA), is presented, where a simulated annealing algorithm is combined with local optimization methods. First, its general procedure and mathematical convergence properties are described. Then, its two example applications are presented, namely, optimization of hidden Markov models for visual speech recognition and optimization of radial basis function networks for pattern classification, in order to show how the HSA algorithm can be successfully adopted for solving real-world problems effectively. As an appendix, the source code for multi-dimensional Cauchy random number generation is provided, which is essential for implementation of the presented method.

UR - http://www.scopus.com/inward/record.url?scp=84885446394&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84885446394&partnerID=8YFLogxK

U2 - 10.1007/978-3-642-30504-7_16

DO - 10.1007/978-3-642-30504-7_16

M3 - Chapter

AN - SCOPUS:84885446394

SN - 9783642305030

T3 - Intelligent Systems Reference Library

SP - 395

EP - 422

BT - Handbook of Optimization

A2 - Zelinka, Ivan

A2 - Snasel, Vaclav

A2 - Abraham, Ajith

ER -

Lee JS, Park CH, Ebrahimi T. Theory and Applications of Hybrid Simulated Annealing. In Zelinka I, Snasel V, Abraham A, editors, Handbook of Optimization: From Classical to Modern Approach. 2013. p. 395-422. (Intelligent Systems Reference Library). https://doi.org/10.1007/978-3-642-30504-7_16