Trace Ratio Optimization for High-Dimensional Multi-Class Discrimination

Jeongyoun Ahn, Hee Cheol Chung, Yongho Jeon

Research output: Contribution to journalArticlepeer-review

Abstract

In multi-class discrimination with high-dimensional data, identifying a lower-dimensional subspace with maximum class separation is crucial. We propose a new optimization criterion for finding such a discriminant subspace, which is the ratio of two traces: the trace of between-class scatter matrix and the trace of within-class scatter matrix. Since this problem is not well-defined for high-dimensional data, we propose to regularize the within trace and maximize the between trace. A careful investigation reveals that this optimization has an innate connection to the eigenvalue decomposition of an indefinite matrix. For the sake of better interpretability of the classifier, we also consider a sparse estimation via a group-wise soft-thresholding. Interesting relationships between the proposed method and some classical methods such as Fisher’s linear discriminant analysis and its variants are discussed. Empirical examples with simulated and real datasets suggest that the proposed method works well and is often better than some existing approaches in a wide range of problems, with respect to both variable selectivity and classification accuracy. Supplementary files for this article are available online.

Original languageEnglish
Pages (from-to)192-203
Number of pages12
JournalJournal of Computational and Graphical Statistics
Volume30
Issue number1
DOIs
Publication statusPublished - 2021

Bibliographical note

Funding Information:
Jeon?s research was supported by Basic Science Research Program of the National Research Foundation of Korea (2019R1A2C100597912) funded by the Korean Government.

Publisher Copyright:
© 2020 American Statistical Association, Institute of Mathematical Statistics, and Interface Foundation of North America.

All Science Journal Classification (ASJC) codes

  • Statistics and Probability
  • Discrete Mathematics and Combinatorics
  • Statistics, Probability and Uncertainty

Fingerprint Dive into the research topics of 'Trace Ratio Optimization for High-Dimensional Multi-Class Discrimination'. Together they form a unique fingerprint.

Cite this