A Low-Memory Learning Formulation for a Kernel-and-Range Network

Huiping Zhuang, Zhiping Lin, Kar Ann Toh

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Recently, a learning method based on the kernel and the range space projections has been introduced. This method has been applied to learn the multilayer network analytically with interpretable relationships among the weight matrices. However, the learning method carries a high-memory demand during training. In this study, a low-memory formulation is proposed to address this issue of high-memory demand. The developed method is inspired by a recursive implementation of the Moore-Penrose inverse and is shown to be mathematically equivalent to the original batch learning. Next, we further improved our proposed low-memory formulation to annul the potential divergence caused by rounding errors. The regression and classification behaviors of the proposed learning method are demonstrated using both synthetic and benchmark datasets. Our experiments confirm that the proposed formulation consumes significantly lower memory.

Original languageEnglish
Title of host publication2019 International Joint Conference on Neural Networks, IJCNN 2019
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781728119854
DOIs
Publication statusPublished - 2019 Jul
Event2019 International Joint Conference on Neural Networks, IJCNN 2019 - Budapest, Hungary
Duration: 2019 Jul 142019 Jul 19

Publication series

NameProceedings of the International Joint Conference on Neural Networks
Volume2019-July

Conference

Conference2019 International Joint Conference on Neural Networks, IJCNN 2019
CountryHungary
CityBudapest
Period19/7/1419/7/19

All Science Journal Classification (ASJC) codes

  • Software
  • Artificial Intelligence

Fingerprint Dive into the research topics of 'A Low-Memory Learning Formulation for a Kernel-and-Range Network'. Together they form a unique fingerprint.

Cite this