SSDStreamer: Specializing I/O Stack for Large-Scale Machine Learning

Jonghyun Bae; Hakbeom Jang; Jeonghun Gong; Wenjing Jin; Shine Kim; Jaeyoung Jang; Tae Jun Ham; Jinkyu Jeong; Jae W. Lee

doi:10.1109/MM.2019.2930497

SSDStreamer: Specializing I/O Stack for Large-Scale Machine Learning

Jonghyun Bae, Hakbeom Jang, Jeonghun Gong, Wenjing Jin, Shine Kim, Jaeyoung Jang, Tae Jun Ham, Jinkyu Jeong, Jae W. Lee

Research output: Contribution to journal › Article › peer-review

Abstract

This article presents SSDStreamer, an SSD-based caching system for large-scale machine learning. By using DRAM as stream buffer, instead of an upper-level cache, SSDStreamer significantly outperforms state-of-the-art multilevel caching systems on Apache Spark, while requiring much less DRAM capacity.

Original language	English
Article number	8770099
Pages (from-to)	73-81
Number of pages	9
Journal	IEEE Micro
Volume	39
Issue number	5
DOIs	https://doi.org/10.1109/MM.2019.2930497
Publication status	Published - 2019 Sept 1

Bibliographical note

Funding Information:
This work was supported in part by the Research Resettlement Fund for the new faculty of Seoul National University; in part by a research grant from Samsung Electronics, National Research Foundation of Korea grant funded by the Ministry of Science, ICT & Future Planning (PE Class Heterogeneous High Performance Computer Development, NRF- 2016M3C4A7952587); and in part by the Competency Development Program for Industry Specialists of the Korean Ministry of Trade, Industry and Energy (MOTIE), operated by the Korea Institute for Advancement of Technology (KIAT) (No. N0001883, HRD Program for Intelligent Semiconductor Industry).

Funding Information:
This work was supported in part by the Research Resettlement Fund for the new faculty of Seoul National University; in part by a research grant from Samsung Electronics, National Research Foundation of Korea grant funded by the Ministry of Science, ICT & Future Planning (PE Class Heterogeneous High Performance Computer Development, NRF-2016M3C4A7952587); and in part by the Competency Development Program for Industry Specialists of the Korean Ministry of Trade, Industry and Energy (MOTIE), operated by the Korea Institute for Advancement of Technology (KIAT) (No. N0001883, HRD Program for Intelligent Semiconductor Industry).

Publisher Copyright:
© 1981-2012 IEEE.

All Science Journal Classification (ASJC) codes

Software
Hardware and Architecture
Electrical and Electronic Engineering

Access to Document

10.1109/MM.2019.2930497

Cite this

@article{986b65940a6e4b7b903c7f75da6efc9a,

title = "SSDStreamer: Specializing I/O Stack for Large-Scale Machine Learning",

abstract = "This article presents SSDStreamer, an SSD-based caching system for large-scale machine learning. By using DRAM as stream buffer, instead of an upper-level cache, SSDStreamer significantly outperforms state-of-the-art multilevel caching systems on Apache Spark, while requiring much less DRAM capacity.",

author = "Jonghyun Bae and Hakbeom Jang and Jeonghun Gong and Wenjing Jin and Shine Kim and Jaeyoung Jang and Ham, {Tae Jun} and Jinkyu Jeong and Lee, {Jae W.}",

note = "Funding Information: This work was supported in part by the Research Resettlement Fund for the new faculty of Seoul National University; in part by a research grant from Samsung Electronics, National Research Foundation of Korea grant funded by the Ministry of Science, ICT & Future Planning (PE Class Heterogeneous High Performance Computer Development, NRF- 2016M3C4A7952587); and in part by the Competency Development Program for Industry Specialists of the Korean Ministry of Trade, Industry and Energy (MOTIE), operated by the Korea Institute for Advancement of Technology (KIAT) (No. N0001883, HRD Program for Intelligent Semiconductor Industry). Funding Information: This work was supported in part by the Research Resettlement Fund for the new faculty of Seoul National University; in part by a research grant from Samsung Electronics, National Research Foundation of Korea grant funded by the Ministry of Science, ICT & Future Planning (PE Class Heterogeneous High Performance Computer Development, NRF-2016M3C4A7952587); and in part by the Competency Development Program for Industry Specialists of the Korean Ministry of Trade, Industry and Energy (MOTIE), operated by the Korea Institute for Advancement of Technology (KIAT) (No. N0001883, HRD Program for Intelligent Semiconductor Industry). Publisher Copyright: {\textcopyright} 1981-2012 IEEE.",

year = "2019",

month = sep,

day = "1",

doi = "10.1109/MM.2019.2930497",

language = "English",

volume = "39",

pages = "73--81",

journal = "IEEE Micro",

issn = "0272-1732",

publisher = "IEEE Computer Society",

number = "5",

}

TY - JOUR

T1 - SSDStreamer

T2 - Specializing I/O Stack for Large-Scale Machine Learning

AU - Bae, Jonghyun

AU - Jang, Hakbeom

AU - Gong, Jeonghun

AU - Jin, Wenjing

AU - Kim, Shine

AU - Jang, Jaeyoung

AU - Ham, Tae Jun

AU - Jeong, Jinkyu

AU - Lee, Jae W.

N1 - Funding Information: This work was supported in part by the Research Resettlement Fund for the new faculty of Seoul National University; in part by a research grant from Samsung Electronics, National Research Foundation of Korea grant funded by the Ministry of Science, ICT & Future Planning (PE Class Heterogeneous High Performance Computer Development, NRF- 2016M3C4A7952587); and in part by the Competency Development Program for Industry Specialists of the Korean Ministry of Trade, Industry and Energy (MOTIE), operated by the Korea Institute for Advancement of Technology (KIAT) (No. N0001883, HRD Program for Intelligent Semiconductor Industry). Funding Information: This work was supported in part by the Research Resettlement Fund for the new faculty of Seoul National University; in part by a research grant from Samsung Electronics, National Research Foundation of Korea grant funded by the Ministry of Science, ICT & Future Planning (PE Class Heterogeneous High Performance Computer Development, NRF-2016M3C4A7952587); and in part by the Competency Development Program for Industry Specialists of the Korean Ministry of Trade, Industry and Energy (MOTIE), operated by the Korea Institute for Advancement of Technology (KIAT) (No. N0001883, HRD Program for Intelligent Semiconductor Industry). Publisher Copyright: © 1981-2012 IEEE.

PY - 2019/9/1

Y1 - 2019/9/1

N2 - This article presents SSDStreamer, an SSD-based caching system for large-scale machine learning. By using DRAM as stream buffer, instead of an upper-level cache, SSDStreamer significantly outperforms state-of-the-art multilevel caching systems on Apache Spark, while requiring much less DRAM capacity.

AB - This article presents SSDStreamer, an SSD-based caching system for large-scale machine learning. By using DRAM as stream buffer, instead of an upper-level cache, SSDStreamer significantly outperforms state-of-the-art multilevel caching systems on Apache Spark, while requiring much less DRAM capacity.

UR - http://www.scopus.com/inward/record.url?scp=85072190310&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85072190310&partnerID=8YFLogxK

U2 - 10.1109/MM.2019.2930497

DO - 10.1109/MM.2019.2930497

M3 - Article

AN - SCOPUS:85072190310

SN - 0272-1732

VL - 39

SP - 73

EP - 81

JO - IEEE Micro

JF - IEEE Micro

IS - 5

M1 - 8770099

ER -

SSDStreamer: Specializing I/O Stack for Large-Scale Machine Learning

Abstract

Bibliographical note

All Science Journal Classification (ASJC) codes

Access to Document

Other files and links

Fingerprint

Cite this