Recently, a learning method based on the kernel and the range space projections has been proposed. This method has been applied to learn the multilayer network analytically with interpretable relationships among the weight matrices. However, the bulk matrix based formulation suffers from a high memory demand during network learning. In this study, a low-memory resolution is proposed to address the memory demanding problem. Essentially, the bulk matrix operations are implemented by a low-memory formulation in which only one training sample is processed at a time. Such a formulation is proved to be mathematically equivalent to the original batch learning version. We also point out that the rounding errors in computing systems could hinder the performance of the proposed formulation. This formulation is then robustified by introducing a regularization technique with the cost of an additional but negligible memory usage. Our experiments show that the proposed low-memory resolution can indeed tremendously reduce the memory consumption while maintaining reasonably good performances in both regression and classification tasks.
Bibliographical noteFunding Information:
This work was partially supported by Nanyang Technological University Research Scholarships. This research was also partially supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education, Science and Technology (Grant number: NRF-2018R1D1A1A09081956 ).
All Science Journal Classification (ASJC) codes
- Control and Systems Engineering
- Signal Processing
- Computer Networks and Communications
- Applied Mathematics