GradPIM: A Practical Processing-in-DRAM Architecture for Gradient Descent

Heesu Kim, Hanmin Park, Taehyun Kim, Kwanheum Cho, Eojin Lee, Soojung Ryu, Hyuk Jae Lee, Kiyoung Choi, Jinho Lee

Research output: Chapter in Book/Report/Conference proceedingConference contribution

9 Citations (Scopus)

Abstract

In this paper, we present GradPIM, a processingin-memory architecture which accelerates parameter updates of deep neural networks training. As one of processing-in-memory techniques that could be realized in the near future, we propose an incremental, simple architectural design that does not invade the existing memory protocol. Extending DDR4 SDRAM to utilize bank-group parallelism makes our operation designs in processing-in-memory (PIM) module efficient in terms of hardware cost and performance. Our experimental results show that the proposed architecture can improve the performance of DNN training and greatly reduce memory bandwidth requirement while posing only a minimal amount of overhead to the protocol and DRAM area.

Original languageEnglish
Title of host publicationProceeding - 27th IEEE International Symposium on High Performance Computer Architecture, HPCA 2021
PublisherIEEE Computer Society
Pages249-262
Number of pages14
ISBN (Electronic)9780738123370
DOIs
Publication statusPublished - 2021 Feb
Event27th Annual IEEE International Symposium on High Performance Computer Architecture, HPCA 2021 - Virtual, Seoul, Korea, Republic of
Duration: 2021 Feb 272021 Mar 1

Publication series

NameProceedings - International Symposium on High-Performance Computer Architecture
Volume2021-February
ISSN (Print)1530-0897

Conference

Conference27th Annual IEEE International Symposium on High Performance Computer Architecture, HPCA 2021
Country/TerritoryKorea, Republic of
CityVirtual, Seoul
Period21/2/2721/3/1

Bibliographical note

Funding Information:
This work was supported by National Research Foundation of Korea (NRF) grant funded by the Korea government(MSIT) (No.2020R1F1A1074472) and R&D program of MOTIE/KEIT [No. 10077609, Developing Processor-Memory-Storage Integrated Architecture for Low Power, High Performance Big Data Servers] and Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No.2020-0-01361, Artificial Intelligence Graduate School Program (Yonsei University)) and Samsung Advanced Institute of Technology, Samsung Electronics Co., Ltd.

Publisher Copyright:
© 2021 IEEE.

All Science Journal Classification (ASJC) codes

  • Hardware and Architecture

Fingerprint

Dive into the research topics of 'GradPIM: A Practical Processing-in-DRAM Architecture for Gradient Descent'. Together they form a unique fingerprint.

Cite this