Exploring fault-tolerant erasure codes for scalable all-flash array clusters

Sungjoon Koh, Jie Zhang, Miryeong Kwon, Jungyeon Yoon, David Donofrio, Nam Sung Kim, Myoungsoo Jung

Research output: Contribution to journalArticlepeer-review

1 Citation (Scopus)

Abstract

Large-scale systems with all-flash arrays have become increasingly common in many computing segments. To make such systems resilient, we can adopt erasure coding such as Reed-Solomon (RS) code as an alternative to replication because erasure coding incurs a significantly lower storage overhead than replication. To understand the impact of using erasure coding on the system performance and other system aspects such as CPU utilization and network traffic, we build a storage cluster that consists of approximately 100 processor cores with more than 50 high-performance solid-state drives (SSDs), and evaluate the cluster with a popular open-source distributed parallel file system, called Ceph. Specifically, we analyze the behaviors of a system adopting erasure coding from the following five viewpoints, and compare with those of another system using replication: (1) storage system I/O performance; (2) computing and software overheads; (3) I/O amplification; (4) network traffic among storage nodes, and (5) impact of physical data layout on performance of RS-coded SSD arrays. For all these analyses, we examine two representative RS configurations, used by Google file systems, and compare them with triple replication employed by a typical parallel file system as a default fault tolerance mechanism. Lastly, we collect 96 block-level traces from the cluster and release them to the public domain for the use of other researchers.

Original languageEnglish
Article number8565994
Pages (from-to)1312-1330
Number of pages19
JournalIEEE Transactions on Parallel and Distributed Systems
Volume30
Issue number6
DOIs
Publication statusPublished - 2019 Jun 1

Bibliographical note

Funding Information:
This is a full version of a conference paper [40]. In this work, we completely revised all the previous evaluation results from scratch by replacing a block interface Ceph with filesystem-enabled Ceph, and evaluated our all-flash array clusters with an on-line erasure coding mechanism by executing diverse real application scenario [20]. This research is mainly supported by NRF 2016R1C1B2015312. This work is also supported in part by Yonsei Future Research Grant (2017-22-0105), IITP-2017-2017-0-01015, NRF-2015M3C4A7065645, DOE DE-AC02-05CH 11231, and MemRay grant (2015-11-1731). Dr. Kim is supported in part by NSF 1640196 and SRC/ NRC NERC 2016-NE-2697-A. Myoungsoo Jung is the corresponding author.

Publisher Copyright:
© 2018 IEEE.

All Science Journal Classification (ASJC) codes

  • Signal Processing
  • Hardware and Architecture
  • Computational Theory and Mathematics

Fingerprint

Dive into the research topics of 'Exploring fault-tolerant erasure codes for scalable all-flash array clusters'. Together they form a unique fingerprint.

Cite this