A weight-adjusted voting algorithm for ensembles of classifiers

Hyunjoong Kim, Hyeuk Kim, Hojin Moon, Hongshik Ahn

Research output: Contribution to journalArticle

40 Citations (Scopus)

Abstract

We present a new weighted voting classification ensemble method, called WAVE, that uses two weight vectors: a weight vector of classifiers and a weight vector of instances. The instance weight vector assigns higher weights to observations that are hard to classify. The weight vector of classifiers puts larger weights on classifiers that perform better on hard-to-classify instances. One weight vector is designed to be calculated in conjunction with the other through an iterative procedure. That is, the instances of higher weights play a more important role in determining the weights of classifiers, and vice versa. We proved that the iterated weight vectors converge to the optimal weights which can be directly calculated from the performance matrix of classifiers in an ensemble. The final prediction of the ensemble is obtained by voting using the optimal weight vector of classifiers. To compare the performance between a simple majority voting and the proposed weighted voting, we applied both of the voting methods to bootstrap aggregation and investigated the performance on 28 datasets. The result shows that the proposed weighted voting performs significantly better than the simple majority voting in general.

Original languageEnglish
Pages (from-to)437-449
Number of pages13
JournalJournal of the Korean Statistical Society
Volume40
Issue number4
DOIs
Publication statusPublished - 2011 Dec 1

Fingerprint

Voting
Ensemble
Classifier
Majority Voting
Classify
Ensemble Methods
Iterative Procedure
Bootstrap
Assign
Aggregation
Converge

All Science Journal Classification (ASJC) codes

  • Statistics and Probability

Cite this

Kim, Hyunjoong ; Kim, Hyeuk ; Moon, Hojin ; Ahn, Hongshik. / A weight-adjusted voting algorithm for ensembles of classifiers. In: Journal of the Korean Statistical Society. 2011 ; Vol. 40, No. 4. pp. 437-449.
@article{63b3d92f1b5048d2866556f6dbbe7e80,
title = "A weight-adjusted voting algorithm for ensembles of classifiers",
abstract = "We present a new weighted voting classification ensemble method, called WAVE, that uses two weight vectors: a weight vector of classifiers and a weight vector of instances. The instance weight vector assigns higher weights to observations that are hard to classify. The weight vector of classifiers puts larger weights on classifiers that perform better on hard-to-classify instances. One weight vector is designed to be calculated in conjunction with the other through an iterative procedure. That is, the instances of higher weights play a more important role in determining the weights of classifiers, and vice versa. We proved that the iterated weight vectors converge to the optimal weights which can be directly calculated from the performance matrix of classifiers in an ensemble. The final prediction of the ensemble is obtained by voting using the optimal weight vector of classifiers. To compare the performance between a simple majority voting and the proposed weighted voting, we applied both of the voting methods to bootstrap aggregation and investigated the performance on 28 datasets. The result shows that the proposed weighted voting performs significantly better than the simple majority voting in general.",
author = "Hyunjoong Kim and Hyeuk Kim and Hojin Moon and Hongshik Ahn",
year = "2011",
month = "12",
day = "1",
doi = "10.1016/j.jkss.2011.03.002",
language = "English",
volume = "40",
pages = "437--449",
journal = "Journal of the Korean Statistical Society",
issn = "1226-3192",
publisher = "Korean Statistical Society",
number = "4",

}

A weight-adjusted voting algorithm for ensembles of classifiers. / Kim, Hyunjoong; Kim, Hyeuk; Moon, Hojin; Ahn, Hongshik.

In: Journal of the Korean Statistical Society, Vol. 40, No. 4, 01.12.2011, p. 437-449.

Research output: Contribution to journalArticle

TY - JOUR

T1 - A weight-adjusted voting algorithm for ensembles of classifiers

AU - Kim, Hyunjoong

AU - Kim, Hyeuk

AU - Moon, Hojin

AU - Ahn, Hongshik

PY - 2011/12/1

Y1 - 2011/12/1

N2 - We present a new weighted voting classification ensemble method, called WAVE, that uses two weight vectors: a weight vector of classifiers and a weight vector of instances. The instance weight vector assigns higher weights to observations that are hard to classify. The weight vector of classifiers puts larger weights on classifiers that perform better on hard-to-classify instances. One weight vector is designed to be calculated in conjunction with the other through an iterative procedure. That is, the instances of higher weights play a more important role in determining the weights of classifiers, and vice versa. We proved that the iterated weight vectors converge to the optimal weights which can be directly calculated from the performance matrix of classifiers in an ensemble. The final prediction of the ensemble is obtained by voting using the optimal weight vector of classifiers. To compare the performance between a simple majority voting and the proposed weighted voting, we applied both of the voting methods to bootstrap aggregation and investigated the performance on 28 datasets. The result shows that the proposed weighted voting performs significantly better than the simple majority voting in general.

AB - We present a new weighted voting classification ensemble method, called WAVE, that uses two weight vectors: a weight vector of classifiers and a weight vector of instances. The instance weight vector assigns higher weights to observations that are hard to classify. The weight vector of classifiers puts larger weights on classifiers that perform better on hard-to-classify instances. One weight vector is designed to be calculated in conjunction with the other through an iterative procedure. That is, the instances of higher weights play a more important role in determining the weights of classifiers, and vice versa. We proved that the iterated weight vectors converge to the optimal weights which can be directly calculated from the performance matrix of classifiers in an ensemble. The final prediction of the ensemble is obtained by voting using the optimal weight vector of classifiers. To compare the performance between a simple majority voting and the proposed weighted voting, we applied both of the voting methods to bootstrap aggregation and investigated the performance on 28 datasets. The result shows that the proposed weighted voting performs significantly better than the simple majority voting in general.

UR - http://www.scopus.com/inward/record.url?scp=82855178868&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=82855178868&partnerID=8YFLogxK

U2 - 10.1016/j.jkss.2011.03.002

DO - 10.1016/j.jkss.2011.03.002

M3 - Article

VL - 40

SP - 437

EP - 449

JO - Journal of the Korean Statistical Society

JF - Journal of the Korean Statistical Society

SN - 1226-3192

IS - 4

ER -