Capacity of clustered distributed storage

Jy Yong Sohn; Beongjun Choi; Sung Whan Yoon; Jaekyun Moon

doi:10.1109/ICC.2017.7997392

Capacity of clustered distributed storage

Jy Yong Sohn, Beongjun Choi, Sung Whan Yoon, Jaekyun Moon

Department of Applied Statistics

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

A new system model reflecting the clustered structure of distributed storage is suggested to investigate bandwidth requirements for repairing failed storage nodes. Large data centers with multiple racks/disks or local networks of storage devices (e.g. sensor network) are good applications of the suggested cluster-based model. In realistic scenarios involving clustered storage structures, repairing storage nodes using intact nodes residing in other clusters is more bandwidth-consuming than restoring nodes based on information from intra-cluster nodes. Therefore, it is important to differentiate between intra-cluster repair bandwidth and cross-cluster repair bandwidth in modeling distributed storage. Capacity of the suggested model is obtained as a function of fundamental resources of distributed storage systems, namely, storage capacity, intra-cluster repair bandwidth and cross-cluster repair bandwidth. Based on the capacity expression, feasible sets of required resources which enable reliable storage are analyzed. It is shown that the cross-cluster traffic can be minimized to zero (i.e., local repair within a cluster becomes possible) by allowing extra resources on storage capacity and intra-cluster repair bandwidth, according to a law specified in a closed-form. Moreover, trade-off between cross-cluster traffic and intra-cluster traffic is observed for sufficiently large storage capacity.

Original language	English
Title of host publication	2017 IEEE International Conference on Communications, ICC 2017
Editors	Merouane Debbah, David Gesbert, Abdelhamid Mellouk
Publisher	Institute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)	9781467389990
DOIs	https://doi.org/10.1109/ICC.2017.7997392
Publication status	Published - 2017 Jul 28
Event	2017 IEEE International Conference on Communications, ICC 2017 - Paris, France Duration: 2017 May 21 → 2017 May 25

Publication series

Name	IEEE International Conference on Communications
ISSN (Print)	1550-3607

Other

Other	2017 IEEE International Conference on Communications, ICC 2017
Country/Territory	France
City	Paris
Period	17/5/21 → 17/5/25

Bibliographical note

Funding Information:
This work is in part supported by the National Research Foundation of Korea under Grant No. 2016R1A2B4011298, and in part supported by the ICT R&D program of MSIP/IITP [2016-0-00563, Research on Adaptive Machine Learning Technology Development for Intelligent Autonomous Digital Companion].

Publisher Copyright:
© 2017 IEEE.

All Science Journal Classification (ASJC) codes

Computer Networks and Communications
Electrical and Electronic Engineering

Access to Document

10.1109/ICC.2017.7997392

Cite this

@inproceedings{b5a668ee0bec4b499e9e75e2209e4833,

title = "Capacity of clustered distributed storage",

abstract = "A new system model reflecting the clustered structure of distributed storage is suggested to investigate bandwidth requirements for repairing failed storage nodes. Large data centers with multiple racks/disks or local networks of storage devices (e.g. sensor network) are good applications of the suggested cluster-based model. In realistic scenarios involving clustered storage structures, repairing storage nodes using intact nodes residing in other clusters is more bandwidth-consuming than restoring nodes based on information from intra-cluster nodes. Therefore, it is important to differentiate between intra-cluster repair bandwidth and cross-cluster repair bandwidth in modeling distributed storage. Capacity of the suggested model is obtained as a function of fundamental resources of distributed storage systems, namely, storage capacity, intra-cluster repair bandwidth and cross-cluster repair bandwidth. Based on the capacity expression, feasible sets of required resources which enable reliable storage are analyzed. It is shown that the cross-cluster traffic can be minimized to zero (i.e., local repair within a cluster becomes possible) by allowing extra resources on storage capacity and intra-cluster repair bandwidth, according to a law specified in a closed-form. Moreover, trade-off between cross-cluster traffic and intra-cluster traffic is observed for sufficiently large storage capacity.",

author = "Sohn, {Jy Yong} and Beongjun Choi and Yoon, {Sung Whan} and Jaekyun Moon",

note = "Funding Information: This work is in part supported by the National Research Foundation of Korea under Grant No. 2016R1A2B4011298, and in part supported by the ICT R&D program of MSIP/IITP [2016-0-00563, Research on Adaptive Machine Learning Technology Development for Intelligent Autonomous Digital Companion]. Publisher Copyright: {\textcopyright} 2017 IEEE.; 2017 IEEE International Conference on Communications, ICC 2017 ; Conference date: 21-05-2017 Through 25-05-2017",

year = "2017",

month = jul,

day = "28",

doi = "10.1109/ICC.2017.7997392",

language = "English",

series = "IEEE International Conference on Communications",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

editor = "Merouane Debbah and David Gesbert and Abdelhamid Mellouk",

booktitle = "2017 IEEE International Conference on Communications, ICC 2017",

address = "United States",

}

Sohn, JY, Choi, B, Yoon, SW & Moon, J 2017, Capacity of clustered distributed storage. in M Debbah, D Gesbert & A Mellouk (eds), 2017 IEEE International Conference on Communications, ICC 2017., 7997392, IEEE International Conference on Communications, Institute of Electrical and Electronics Engineers Inc., 2017 IEEE International Conference on Communications, ICC 2017, Paris, France, 17/5/21. https://doi.org/10.1109/ICC.2017.7997392

Capacity of clustered distributed storage. / Sohn, Jy Yong; Choi, Beongjun; Yoon, Sung Whan et al.

2017 IEEE International Conference on Communications, ICC 2017. ed. / Merouane Debbah; David Gesbert; Abdelhamid Mellouk. Institute of Electrical and Electronics Engineers Inc., 2017. 7997392 (IEEE International Conference on Communications).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Capacity of clustered distributed storage

AU - Sohn, Jy Yong

AU - Choi, Beongjun

AU - Yoon, Sung Whan

AU - Moon, Jaekyun

N1 - Funding Information: This work is in part supported by the National Research Foundation of Korea under Grant No. 2016R1A2B4011298, and in part supported by the ICT R&D program of MSIP/IITP [2016-0-00563, Research on Adaptive Machine Learning Technology Development for Intelligent Autonomous Digital Companion]. Publisher Copyright: © 2017 IEEE.

PY - 2017/7/28

Y1 - 2017/7/28

N2 - A new system model reflecting the clustered structure of distributed storage is suggested to investigate bandwidth requirements for repairing failed storage nodes. Large data centers with multiple racks/disks or local networks of storage devices (e.g. sensor network) are good applications of the suggested cluster-based model. In realistic scenarios involving clustered storage structures, repairing storage nodes using intact nodes residing in other clusters is more bandwidth-consuming than restoring nodes based on information from intra-cluster nodes. Therefore, it is important to differentiate between intra-cluster repair bandwidth and cross-cluster repair bandwidth in modeling distributed storage. Capacity of the suggested model is obtained as a function of fundamental resources of distributed storage systems, namely, storage capacity, intra-cluster repair bandwidth and cross-cluster repair bandwidth. Based on the capacity expression, feasible sets of required resources which enable reliable storage are analyzed. It is shown that the cross-cluster traffic can be minimized to zero (i.e., local repair within a cluster becomes possible) by allowing extra resources on storage capacity and intra-cluster repair bandwidth, according to a law specified in a closed-form. Moreover, trade-off between cross-cluster traffic and intra-cluster traffic is observed for sufficiently large storage capacity.

AB - A new system model reflecting the clustered structure of distributed storage is suggested to investigate bandwidth requirements for repairing failed storage nodes. Large data centers with multiple racks/disks or local networks of storage devices (e.g. sensor network) are good applications of the suggested cluster-based model. In realistic scenarios involving clustered storage structures, repairing storage nodes using intact nodes residing in other clusters is more bandwidth-consuming than restoring nodes based on information from intra-cluster nodes. Therefore, it is important to differentiate between intra-cluster repair bandwidth and cross-cluster repair bandwidth in modeling distributed storage. Capacity of the suggested model is obtained as a function of fundamental resources of distributed storage systems, namely, storage capacity, intra-cluster repair bandwidth and cross-cluster repair bandwidth. Based on the capacity expression, feasible sets of required resources which enable reliable storage are analyzed. It is shown that the cross-cluster traffic can be minimized to zero (i.e., local repair within a cluster becomes possible) by allowing extra resources on storage capacity and intra-cluster repair bandwidth, according to a law specified in a closed-form. Moreover, trade-off between cross-cluster traffic and intra-cluster traffic is observed for sufficiently large storage capacity.

UR - http://www.scopus.com/inward/record.url?scp=85028351235&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85028351235&partnerID=8YFLogxK

U2 - 10.1109/ICC.2017.7997392

DO - 10.1109/ICC.2017.7997392

M3 - Conference contribution

AN - SCOPUS:85028351235

T3 - IEEE International Conference on Communications

BT - 2017 IEEE International Conference on Communications, ICC 2017

A2 - Debbah, Merouane

A2 - Gesbert, David

A2 - Mellouk, Abdelhamid

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2017 IEEE International Conference on Communications, ICC 2017

Y2 - 21 May 2017 through 25 May 2017

ER -

Capacity of clustered distributed storage

Abstract

Publication series

Other

Bibliographical note

All Science Journal Classification (ASJC) codes

Access to Document

Other files and links

Fingerprint

Cite this