The Nebula Benchmark Suite: Implications of Lightweight Neural Networks

Bogil Kim, Sungjae Lee, Chanho Park, Hyeonjin Kim, William Jinho Song

Research output: Contribution to journalArticlepeer-review

Abstract

This paper presents a benchmark suite named Nebula that implements lightweight neural network benchmarks. Recent neural networks tend to form deeper and sizable networks to enhance accuracy and applicability. However, the massive volume of heavy networks makes them highly challenging to use in conventional research environments such as microarchitecture simulators. We notice that neural network computations are mainly comprised of matrix and vector calculations that repeat on multi-dimensional data. This observation motivates us to develop a variable-sized neural network benchmark suite that provides users with options to select appropriate size of benchmarks for different research purposes or experiment conditions. Inspired by the implementations of well-known benchmarks such as PARSEC and SPLASH suites, Nebula offers various size options from large to small datasets for diverse types of neural networks. The Nebula benchmark suite is comprised of seven representative neural networks built on a C++ framework. This paper presents a methodology to develop the variable-sized neural network benchmarks, and their performance and characteristics are evaluated based on hardware measurements. The results demonstrate that the Nebula benchmarks reduce execution time as much as 25x while preserving similar architectural behaviors as the full-fledged neural networks.
Original languageEnglish
Number of pages15
JournalIEEE Transactions on Computers
DOIs
Publication statusAccepted/In press - 2020

All Science Journal Classification (ASJC) codes

  • Software

Fingerprint Dive into the research topics of 'The Nebula Benchmark Suite: Implications of Lightweight Neural Networks'. Together they form a unique fingerprint.

Cite this