We present a new weighted voting classification ensemble method, called WAVE, that uses two weight vectors: a weight vector of classifiers and a weight vector of instances. The instance weight vector assigns higher weights to observations that are hard to classify. The weight vector of classifiers puts larger weights on classifiers that perform better on hard-to-classify instances. One weight vector is designed to be calculated in conjunction with the other through an iterative procedure. That is, the instances of higher weights play a more important role in determining the weights of classifiers, and vice versa. We proved that the iterated weight vectors converge to the optimal weights which can be directly calculated from the performance matrix of classifiers in an ensemble. The final prediction of the ensemble is obtained by voting using the optimal weight vector of classifiers. To compare the performance between a simple majority voting and the proposed weighted voting, we applied both of the voting methods to bootstrap aggregation and investigated the performance on 28 datasets. The result shows that the proposed weighted voting performs significantly better than the simple majority voting in general.
Bibliographical noteFunding Information:
The authors gratefully acknowledge the many helpful suggestions of anonymous reviewers. The authors are also grateful to Professor Chong Jin Park of California State University at San Diego for the contribution of the weighted voting scheme. This work was partly supported by Basic Science Research program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education, Science, and Technology ( 2009-0072019 ).
All Science Journal Classification (ASJC) codes
- Statistics and Probability