Recently, a learning method based on the kernel and the range space projections has been introduced. This method has been applied to learn the multilayer network analytically with interpretable relationships among the weight matrices. However, the learning method carries a high-memory demand during training. In this study, a low-memory formulation is proposed to address this issue of high-memory demand. The developed method is inspired by a recursive implementation of the Moore-Penrose inverse and is shown to be mathematically equivalent to the original batch learning. Next, we further improved our proposed low-memory formulation to annul the potential divergence caused by rounding errors. The regression and classification behaviors of the proposed learning method are demonstrated using both synthetic and benchmark datasets. Our experiments confirm that the proposed formulation consumes significantly lower memory.
|Title of host publication||2019 International Joint Conference on Neural Networks, IJCNN 2019|
|Publisher||Institute of Electrical and Electronics Engineers Inc.|
|Publication status||Published - 2019 Jul|
|Event||2019 International Joint Conference on Neural Networks, IJCNN 2019 - Budapest, Hungary|
Duration: 2019 Jul 14 → 2019 Jul 19
|Name||Proceedings of the International Joint Conference on Neural Networks|
|Conference||2019 International Joint Conference on Neural Networks, IJCNN 2019|
|Period||19/7/14 → 19/7/19|
Bibliographical notePublisher Copyright:
© 2019 IEEE.
All Science Journal Classification (ASJC) codes
- Artificial Intelligence