Bayesian optimization-based global optimal rank selection for compression of convolutional neural networks

Taehyeon Kim, Jieun Lee, Yoonsik Choe

Research output: Contribution to journalArticle

Abstract

Recently, convolutional neural network (CNN) compression via low-rank decomposition has achieved remarkable performance. Finding the optimal rank is a crucial problem because rank is the only hyperparameter for controlling computational complexity and accuracy in compressed CNNs. In this paper, we propose a global optimal rank selection method based on Bayesian optimization (BayesOpt), which is a machine learning based global optimization technique. By utilizing both a simple objective function and a proper optimization scheme, the proposed method produces a global optimal rank that provides a good trade-off between computational complexity and accuracy degradation. In addition, our method also reflects the correlation of each rank in multi-rank selection, and is able to flexibly yield an optimal rank with a given fixed compression ratio. Experimental results indicate that the proposed algorithm can identify the global optimal rank regardless of the huge size of dataset or the various structural features of CNNs. In all experiments on multi-rank selection, the proposed method produces the rank with higher accuracy and lower computational complexity than the state-of-the-art rank selection method, variational Bayesian matrix factorization (VBMF).

Original languageEnglish
Article number8964358
Pages (from-to)17605-17618
Number of pages14
JournalIEEE Access
Volume8
DOIs
Publication statusPublished - 2020

All Science Journal Classification (ASJC) codes

  • Computer Science(all)
  • Materials Science(all)
  • Engineering(all)

Fingerprint Dive into the research topics of 'Bayesian optimization-based global optimal rank selection for compression of convolutional neural networks'. Together they form a unique fingerprint.

  • Cite this