PIMCaffe: Functional Evaluation of a Machine Learning Framework for In-Memory Neural Processing Unit

Won Jeon, Jiwon Lee, Dongseok Kang, Hongju Kal, Won Woo Ro

Research output: Contribution to journalArticlepeer-review

1 Citation (Scopus)


The large amount of memory usage in recent machine learning applications imposes a significant system burden with respect to power and processing speed. To cope with such a problem, Processing-In-Memory (PIM) techniques can be applied as an alternative solution. Especially, the recommendation system, which is one of the major machine learning applications used in data centers, requires a large memory capacity and therefore represents a suitable candidate application that could be helped by the PIM technique. In this paper, we introduce a machine learning framework, PIMCaffe, designed for in-memory neural processing units and its evaluation environment. PIMCaffe consists of two components: A Caffe2-based deep learning framework that supports PIM acceleration and a PIM-emulating hardware platform. We develop a suite of functions, libraries, application programming interfaces, and a device driver to support the framework. In addition, we implement a prototype Neural Processing Unit (NPU) in PIMCaffe to evaluate the performance of our platform with machine learning applications. Our prototype NPU design includes a vector processor for parallel vector processing and a systolic array unit for matrix multiplication. Using the proposed software framework, we perform a detailed analysis of the in-memory neural processing unit. PIMCaffe supports evaluations of recommendation systems and various convolutional neural network models on the in-memory neural processing unit. PIMCaffe with the NPU shows up to $2.26\times $ , $5.99\times $ , and $1.71\times $ speedup, compared to the ARM Cortex-A53 CPU, for the recommendation system, AlexNet, and ResNet-50, respectively.

Original languageEnglish
Article number9469808
Pages (from-to)96629-96640
Number of pages12
JournalIEEE Access
Publication statusPublished - 2021

Bibliographical note

Funding Information:
This work was supported in part by the Institute of Information Communications Technology Planning Evaluation (IITP) grant funded by the Korea Government [Ministry of Science and ICT (MSIT)] (No. 2021-0-00853, Developing Software Platform for Programming of Processing-In-Memory (PIM), 50%), in part by Samsung Electronics Company Ltd., Hwaseong, South Korea, and in part by the Graduate School of Yonsei University Research Scholarship Grants, in 2020.

Publisher Copyright:
© 2013 IEEE.

All Science Journal Classification (ASJC) codes

  • Computer Science(all)
  • Materials Science(all)
  • Engineering(all)
  • Electrical and Electronic Engineering


Dive into the research topics of 'PIMCaffe: Functional Evaluation of a Machine Learning Framework for In-Memory Neural Processing Unit'. Together they form a unique fingerprint.

Cite this