Exploring fault-tolerant erasure codes for scalable all-flash array clusters

Sungjoon Koh, Jie Zhang, Miryeong Kwon, Jungyeon Yoon, David Donofrio, Nam Sung Kim, Myoungsoo Jung

Research output: Contribution to journalArticle

Abstract

Large-scale systems with all-flash arrays have become increasingly common in many computing segments. To make such systems resilient, we can adopt erasure coding such as Reed-Solomon (RS) code as an alternative to replication because erasure coding incurs a significantly lower storage overhead than replication. To understand the impact of using erasure coding on the system performance and other system aspects such as CPU utilization and network traffic, we build a storage cluster that consists of approximately 100 processor cores with more than 50 high-performance solid-state drives (SSDs), and evaluate the cluster with a popular open-source distributed parallel file system, called Ceph. Specifically, we analyze the behaviors of a system adopting erasure coding from the following five viewpoints, and compare with those of another system using replication: (1) storage system I/O performance; (2) computing and software overheads; (3) I/O amplification; (4) network traffic among storage nodes, and (5) impact of physical data layout on performance of RS-coded SSD arrays. For all these analyses, we examine two representative RS configurations, used by Google file systems, and compare them with triple replication employed by a typical parallel file system as a default fault tolerance mechanism. Lastly, we collect 96 block-level traces from the cluster and release them to the public domain for the use of other researchers.

Original languageEnglish
Article number8565994
Pages (from-to)1312-1330
Number of pages19
JournalIEEE Transactions on Parallel and Distributed Systems
Volume30
Issue number6
DOIs
Publication statusPublished - 2019 Jun 1

All Science Journal Classification (ASJC) codes

  • Signal Processing
  • Hardware and Architecture
  • Computational Theory and Mathematics

Fingerprint Dive into the research topics of 'Exploring fault-tolerant erasure codes for scalable all-flash array clusters'. Together they form a unique fingerprint.

  • Cite this

    Koh, S., Zhang, J., Kwon, M., Yoon, J., Donofrio, D., Kim, N. S., & Jung, M. (2019). Exploring fault-tolerant erasure codes for scalable all-flash array clusters. IEEE Transactions on Parallel and Distributed Systems, 30(6), 1312-1330. [8565994]. https://doi.org/10.1109/TPDS.2018.2884722