A framework for empirical classifiers comparison

Mohammad Ridzuan Bin Abdullah, Kar Ann Toh, Dipti Srinivasan

Research output: Chapter in Book/Report/Conference proceedingConference contribution

4 Citations (Scopus)

Abstract

In this paper, we seek to establish a framework for empirical comparison of performance of pattern classifiers, allowing comparisons to be made consistently across different studies. As many as 106 datasets from the University of California, Irvine, Machine Learning Repository were used as comparison benchmarks. The framework provides a clear definition of the experimental setup so that it can be unambiguously reproduced or verified by others. Multiple runs of cross-validation and tuning were employed to minimize the possibility of random effects causing much biases in the results obtained. The metrics used to compare among different classifiers are based solely on simple readings obtained through classification tests. This allows future comparisons to be made readily adaptable for inclusion of new metrics.

Original languageEnglish
Title of host publication2006 1st IEEE Conference on Industrial Electronics and Applications
DOIs
Publication statusPublished - 2006 Dec 1
Event2006 1st IEEE Conference on Industrial Electronics and Applications, ICIEA 2006 - Singapore, Singapore
Duration: 2006 May 242006 May 26

Publication series

Name2006 1st IEEE Conference on Industrial Electronics and Applications

Other

Other2006 1st IEEE Conference on Industrial Electronics and Applications, ICIEA 2006
CountrySingapore
CitySingapore
Period06/5/2406/5/26

Fingerprint

Classifiers
Learning systems
Tuning

All Science Journal Classification (ASJC) codes

  • Electrical and Electronic Engineering
  • Industrial and Manufacturing Engineering

Cite this

Abdullah, M. R. B., Toh, K. A., & Srinivasan, D. (2006). A framework for empirical classifiers comparison. In 2006 1st IEEE Conference on Industrial Electronics and Applications [4026002] (2006 1st IEEE Conference on Industrial Electronics and Applications). https://doi.org/10.1109/ICIEA.2006.257073
Abdullah, Mohammad Ridzuan Bin ; Toh, Kar Ann ; Srinivasan, Dipti. / A framework for empirical classifiers comparison. 2006 1st IEEE Conference on Industrial Electronics and Applications. 2006. (2006 1st IEEE Conference on Industrial Electronics and Applications).
@inproceedings{213eb15ba2604aa8914df08b602e8174,
title = "A framework for empirical classifiers comparison",
abstract = "In this paper, we seek to establish a framework for empirical comparison of performance of pattern classifiers, allowing comparisons to be made consistently across different studies. As many as 106 datasets from the University of California, Irvine, Machine Learning Repository were used as comparison benchmarks. The framework provides a clear definition of the experimental setup so that it can be unambiguously reproduced or verified by others. Multiple runs of cross-validation and tuning were employed to minimize the possibility of random effects causing much biases in the results obtained. The metrics used to compare among different classifiers are based solely on simple readings obtained through classification tests. This allows future comparisons to be made readily adaptable for inclusion of new metrics.",
author = "Abdullah, {Mohammad Ridzuan Bin} and Toh, {Kar Ann} and Dipti Srinivasan",
year = "2006",
month = "12",
day = "1",
doi = "10.1109/ICIEA.2006.257073",
language = "English",
isbn = "078039514X",
series = "2006 1st IEEE Conference on Industrial Electronics and Applications",
booktitle = "2006 1st IEEE Conference on Industrial Electronics and Applications",

}

Abdullah, MRB, Toh, KA & Srinivasan, D 2006, A framework for empirical classifiers comparison. in 2006 1st IEEE Conference on Industrial Electronics and Applications., 4026002, 2006 1st IEEE Conference on Industrial Electronics and Applications, 2006 1st IEEE Conference on Industrial Electronics and Applications, ICIEA 2006, Singapore, Singapore, 06/5/24. https://doi.org/10.1109/ICIEA.2006.257073

A framework for empirical classifiers comparison. / Abdullah, Mohammad Ridzuan Bin; Toh, Kar Ann; Srinivasan, Dipti.

2006 1st IEEE Conference on Industrial Electronics and Applications. 2006. 4026002 (2006 1st IEEE Conference on Industrial Electronics and Applications).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - A framework for empirical classifiers comparison

AU - Abdullah, Mohammad Ridzuan Bin

AU - Toh, Kar Ann

AU - Srinivasan, Dipti

PY - 2006/12/1

Y1 - 2006/12/1

N2 - In this paper, we seek to establish a framework for empirical comparison of performance of pattern classifiers, allowing comparisons to be made consistently across different studies. As many as 106 datasets from the University of California, Irvine, Machine Learning Repository were used as comparison benchmarks. The framework provides a clear definition of the experimental setup so that it can be unambiguously reproduced or verified by others. Multiple runs of cross-validation and tuning were employed to minimize the possibility of random effects causing much biases in the results obtained. The metrics used to compare among different classifiers are based solely on simple readings obtained through classification tests. This allows future comparisons to be made readily adaptable for inclusion of new metrics.

AB - In this paper, we seek to establish a framework for empirical comparison of performance of pattern classifiers, allowing comparisons to be made consistently across different studies. As many as 106 datasets from the University of California, Irvine, Machine Learning Repository were used as comparison benchmarks. The framework provides a clear definition of the experimental setup so that it can be unambiguously reproduced or verified by others. Multiple runs of cross-validation and tuning were employed to minimize the possibility of random effects causing much biases in the results obtained. The metrics used to compare among different classifiers are based solely on simple readings obtained through classification tests. This allows future comparisons to be made readily adaptable for inclusion of new metrics.

UR - http://www.scopus.com/inward/record.url?scp=42749098742&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=42749098742&partnerID=8YFLogxK

U2 - 10.1109/ICIEA.2006.257073

DO - 10.1109/ICIEA.2006.257073

M3 - Conference contribution

SN - 078039514X

SN - 9780780395145

T3 - 2006 1st IEEE Conference on Industrial Electronics and Applications

BT - 2006 1st IEEE Conference on Industrial Electronics and Applications

ER -

Abdullah MRB, Toh KA, Srinivasan D. A framework for empirical classifiers comparison. In 2006 1st IEEE Conference on Industrial Electronics and Applications. 2006. 4026002. (2006 1st IEEE Conference on Industrial Electronics and Applications). https://doi.org/10.1109/ICIEA.2006.257073