Backbone architectures of most binary networks are well-known floating point (FP) architectures such as the ResNet family. Questioning that the architectures designed for FP networks might not be the best for binary networks, we propose to search architectures for binary networks (BNAS) by defining a new search space for binary architectures and a novel search objective. Specifically, based on the cell based search method, we define the new search space of binary layer types, design a new cell template, and rediscover the utility of and propose to use the Zeroise layer instead of using it as a placeholder. The novel search objective diversifies early search to learn better performing binary architectures. We show that our method searches architectures with stable training curves despite the quantization error inherent in binary networks. Quantitative analyses demonstrate that our searched architectures outperform the architectures used in state-of-the-art binary networks and outperform or perform on par with state-of-the-art binary networks that employ various techniques other than architectural changes.
|Title of host publication||Computer Vision – ECCV 2020 - 16th European Conference, Proceedings|
|Editors||Andrea Vedaldi, Horst Bischof, Thomas Brox, Jan-Michael Frahm|
|Publisher||Springer Science and Business Media Deutschland GmbH|
|Number of pages||17|
|Publication status||Published - 2020|
|Event||16th European Conference on Computer Vision, ECCV 2020 - Glasgow, United Kingdom|
Duration: 2020 Aug 23 → 2020 Aug 28
|Name||Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)|
|Conference||16th European Conference on Computer Vision, ECCV 2020|
|Period||20/8/23 → 20/8/28|
Bibliographical noteFunding Information:
Acknowledgement. This work was partly supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No.2019R1 C1C1009283), Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No.2019-0-01842, Artificial Intelligence Graduate School Program (GIST) and No.2019-0-01351, Development of Ultra Low-Power Mobile Deep Learning Semiconductor With Compression/Decompression of Activation/Kernel Data), “GIST Research Institute(GRI) GIST-CNUH research Collaboration” grant funded by the GIST in 2020, and a study on the “HPC Support” Project, supported by the ‘Ministry of Science and ICT’ and NIPA.
© 2020, Springer Nature Switzerland AG.
All Science Journal Classification (ASJC) codes
- Theoretical Computer Science
- Computer Science(all)