Rapid backpropagation learning algorithms

Sung-Bae Cho, Jin H. Kim

Research output: Contribution to journalArticle

8 Citations (Scopus)

Abstract

One of the major drawbacks of the backpropagation algorithm is its slow rate of convergence. Researchers have tried several different approaches to speed up the convergence of backpropagation learning. In this paper, we present those rapid learning methods as three categories, and implement the representative methods of each category: (1) for the numerical method based approach, the Aitken's Δ2 process, (2) for the heuristics based approach, the dynamic adaptation of learning rate, and (3) for the learning strategy based approach, the selective presentation of learning samples. Based on these implementations, the performance is evaluated with experiments and the merits and demerits are briefly discussed.

Original languageEnglish
Pages (from-to)155-175
Number of pages21
JournalCircuits, Systems, and Signal Processing
Volume12
Issue number2
DOIs
Publication statusPublished - 1993 Jun 1

Fingerprint

Backpropagation algorithms
Back-propagation Algorithm
Backpropagation
Learning algorithms
Learning Algorithm
Numerical methods
Dynamic Adaptation
Learning Rate
Learning Strategies
Experiments
Back Propagation
Rate of Convergence
Speedup
Numerical Methods
Heuristics
Experiment
Learning

All Science Journal Classification (ASJC) codes

  • Signal Processing
  • Applied Mathematics

Cite this

@article{1f834145dea84febb18afd17f12c2553,
title = "Rapid backpropagation learning algorithms",
abstract = "One of the major drawbacks of the backpropagation algorithm is its slow rate of convergence. Researchers have tried several different approaches to speed up the convergence of backpropagation learning. In this paper, we present those rapid learning methods as three categories, and implement the representative methods of each category: (1) for the numerical method based approach, the Aitken's Δ2 process, (2) for the heuristics based approach, the dynamic adaptation of learning rate, and (3) for the learning strategy based approach, the selective presentation of learning samples. Based on these implementations, the performance is evaluated with experiments and the merits and demerits are briefly discussed.",
author = "Sung-Bae Cho and Kim, {Jin H.}",
year = "1993",
month = "6",
day = "1",
doi = "10.1007/BF01189872",
language = "English",
volume = "12",
pages = "155--175",
journal = "Circuits, Systems, and Signal Processing",
issn = "0278-081X",
publisher = "Birkhause Boston",
number = "2",

}

Rapid backpropagation learning algorithms. / Cho, Sung-Bae; Kim, Jin H.

In: Circuits, Systems, and Signal Processing, Vol. 12, No. 2, 01.06.1993, p. 155-175.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Rapid backpropagation learning algorithms

AU - Cho, Sung-Bae

AU - Kim, Jin H.

PY - 1993/6/1

Y1 - 1993/6/1

N2 - One of the major drawbacks of the backpropagation algorithm is its slow rate of convergence. Researchers have tried several different approaches to speed up the convergence of backpropagation learning. In this paper, we present those rapid learning methods as three categories, and implement the representative methods of each category: (1) for the numerical method based approach, the Aitken's Δ2 process, (2) for the heuristics based approach, the dynamic adaptation of learning rate, and (3) for the learning strategy based approach, the selective presentation of learning samples. Based on these implementations, the performance is evaluated with experiments and the merits and demerits are briefly discussed.

AB - One of the major drawbacks of the backpropagation algorithm is its slow rate of convergence. Researchers have tried several different approaches to speed up the convergence of backpropagation learning. In this paper, we present those rapid learning methods as three categories, and implement the representative methods of each category: (1) for the numerical method based approach, the Aitken's Δ2 process, (2) for the heuristics based approach, the dynamic adaptation of learning rate, and (3) for the learning strategy based approach, the selective presentation of learning samples. Based on these implementations, the performance is evaluated with experiments and the merits and demerits are briefly discussed.

UR - http://www.scopus.com/inward/record.url?scp=0027308381&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0027308381&partnerID=8YFLogxK

U2 - 10.1007/BF01189872

DO - 10.1007/BF01189872

M3 - Article

AN - SCOPUS:0027308381

VL - 12

SP - 155

EP - 175

JO - Circuits, Systems, and Signal Processing

JF - Circuits, Systems, and Signal Processing

SN - 0278-081X

IS - 2

ER -