LESS: Logging Exploiting SnapShot

Hanseung Sung, Minhwa Jin, Mincheol Shin, Hongchan Roh, Wongi Choi, Sang Hyun Park

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

The in-memory key-value store provides a persistence method to ensure data durability. The currently provided methods are either to create a snapshot file of a current dataset or to write the log of the performed command in the log file. However, the snapshot method has a risk of data loss and append only logging method cause a system failure due to an increase in log file size. To prevent excessive AOF file size growth, the in-memory key-value store provides a reconstruction method, but also a performance degradation and excessive memory usage occur. In this paper, we propose a new persistence method for effective memory usage and throughput. The new approach is called Logging Exploiting SnapShot (LESS). LESS is a method that combines the advantages of a snapshot using low memory usage and the benefits of an append only logging method that guarantees data persistence. We implemented LESS on Redis and conducted experiments. A benchmark test demonstrated that the proposed approach reduces the maximum memory usage by 57% and it is 2.7 times faster than the original approach. Overall, the experimental results showed that LESS is effective for Redis.

Original languageEnglish
Title of host publication2019 IEEE International Conference on Big Data and Smart Computing, BigComp 2019 - Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781538677896
DOIs
Publication statusPublished - 2019 Apr 1
Event2019 IEEE International Conference on Big Data and Smart Computing, BigComp 2019 - Kyoto, Japan
Duration: 2019 Feb 272019 Mar 2

Publication series

Name2019 IEEE International Conference on Big Data and Smart Computing, BigComp 2019 - Proceedings

Conference

Conference2019 IEEE International Conference on Big Data and Smart Computing, BigComp 2019
CountryJapan
CityKyoto
Period19/2/2719/3/2

Fingerprint

Data storage equipment
Logging
Durability
Throughput
Degradation
Experiments
Persistence

All Science Journal Classification (ASJC) codes

  • Information Systems and Management
  • Artificial Intelligence
  • Computer Networks and Communications
  • Information Systems

Cite this

Sung, H., Jin, M., Shin, M., Roh, H., Choi, W., & Park, S. H. (2019). LESS: Logging Exploiting SnapShot. In 2019 IEEE International Conference on Big Data and Smart Computing, BigComp 2019 - Proceedings [8679377] (2019 IEEE International Conference on Big Data and Smart Computing, BigComp 2019 - Proceedings). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/BIGCOMP.2019.8679377
Sung, Hanseung ; Jin, Minhwa ; Shin, Mincheol ; Roh, Hongchan ; Choi, Wongi ; Park, Sang Hyun. / LESS : Logging Exploiting SnapShot. 2019 IEEE International Conference on Big Data and Smart Computing, BigComp 2019 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2019. (2019 IEEE International Conference on Big Data and Smart Computing, BigComp 2019 - Proceedings).
@inproceedings{5275b6a8172e447bb6c59cf4c6f87316,
title = "LESS: Logging Exploiting SnapShot",
abstract = "The in-memory key-value store provides a persistence method to ensure data durability. The currently provided methods are either to create a snapshot file of a current dataset or to write the log of the performed command in the log file. However, the snapshot method has a risk of data loss and append only logging method cause a system failure due to an increase in log file size. To prevent excessive AOF file size growth, the in-memory key-value store provides a reconstruction method, but also a performance degradation and excessive memory usage occur. In this paper, we propose a new persistence method for effective memory usage and throughput. The new approach is called Logging Exploiting SnapShot (LESS). LESS is a method that combines the advantages of a snapshot using low memory usage and the benefits of an append only logging method that guarantees data persistence. We implemented LESS on Redis and conducted experiments. A benchmark test demonstrated that the proposed approach reduces the maximum memory usage by 57{\%} and it is 2.7 times faster than the original approach. Overall, the experimental results showed that LESS is effective for Redis.",
author = "Hanseung Sung and Minhwa Jin and Mincheol Shin and Hongchan Roh and Wongi Choi and Park, {Sang Hyun}",
year = "2019",
month = "4",
day = "1",
doi = "10.1109/BIGCOMP.2019.8679377",
language = "English",
series = "2019 IEEE International Conference on Big Data and Smart Computing, BigComp 2019 - Proceedings",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
booktitle = "2019 IEEE International Conference on Big Data and Smart Computing, BigComp 2019 - Proceedings",
address = "United States",

}

Sung, H, Jin, M, Shin, M, Roh, H, Choi, W & Park, SH 2019, LESS: Logging Exploiting SnapShot. in 2019 IEEE International Conference on Big Data and Smart Computing, BigComp 2019 - Proceedings., 8679377, 2019 IEEE International Conference on Big Data and Smart Computing, BigComp 2019 - Proceedings, Institute of Electrical and Electronics Engineers Inc., 2019 IEEE International Conference on Big Data and Smart Computing, BigComp 2019, Kyoto, Japan, 19/2/27. https://doi.org/10.1109/BIGCOMP.2019.8679377

LESS : Logging Exploiting SnapShot. / Sung, Hanseung; Jin, Minhwa; Shin, Mincheol; Roh, Hongchan; Choi, Wongi; Park, Sang Hyun.

2019 IEEE International Conference on Big Data and Smart Computing, BigComp 2019 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2019. 8679377 (2019 IEEE International Conference on Big Data and Smart Computing, BigComp 2019 - Proceedings).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - LESS

T2 - Logging Exploiting SnapShot

AU - Sung, Hanseung

AU - Jin, Minhwa

AU - Shin, Mincheol

AU - Roh, Hongchan

AU - Choi, Wongi

AU - Park, Sang Hyun

PY - 2019/4/1

Y1 - 2019/4/1

N2 - The in-memory key-value store provides a persistence method to ensure data durability. The currently provided methods are either to create a snapshot file of a current dataset or to write the log of the performed command in the log file. However, the snapshot method has a risk of data loss and append only logging method cause a system failure due to an increase in log file size. To prevent excessive AOF file size growth, the in-memory key-value store provides a reconstruction method, but also a performance degradation and excessive memory usage occur. In this paper, we propose a new persistence method for effective memory usage and throughput. The new approach is called Logging Exploiting SnapShot (LESS). LESS is a method that combines the advantages of a snapshot using low memory usage and the benefits of an append only logging method that guarantees data persistence. We implemented LESS on Redis and conducted experiments. A benchmark test demonstrated that the proposed approach reduces the maximum memory usage by 57% and it is 2.7 times faster than the original approach. Overall, the experimental results showed that LESS is effective for Redis.

AB - The in-memory key-value store provides a persistence method to ensure data durability. The currently provided methods are either to create a snapshot file of a current dataset or to write the log of the performed command in the log file. However, the snapshot method has a risk of data loss and append only logging method cause a system failure due to an increase in log file size. To prevent excessive AOF file size growth, the in-memory key-value store provides a reconstruction method, but also a performance degradation and excessive memory usage occur. In this paper, we propose a new persistence method for effective memory usage and throughput. The new approach is called Logging Exploiting SnapShot (LESS). LESS is a method that combines the advantages of a snapshot using low memory usage and the benefits of an append only logging method that guarantees data persistence. We implemented LESS on Redis and conducted experiments. A benchmark test demonstrated that the proposed approach reduces the maximum memory usage by 57% and it is 2.7 times faster than the original approach. Overall, the experimental results showed that LESS is effective for Redis.

UR - http://www.scopus.com/inward/record.url?scp=85064600659&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85064600659&partnerID=8YFLogxK

U2 - 10.1109/BIGCOMP.2019.8679377

DO - 10.1109/BIGCOMP.2019.8679377

M3 - Conference contribution

AN - SCOPUS:85064600659

T3 - 2019 IEEE International Conference on Big Data and Smart Computing, BigComp 2019 - Proceedings

BT - 2019 IEEE International Conference on Big Data and Smart Computing, BigComp 2019 - Proceedings

PB - Institute of Electrical and Electronics Engineers Inc.

ER -

Sung H, Jin M, Shin M, Roh H, Choi W, Park SH. LESS: Logging Exploiting SnapShot. In 2019 IEEE International Conference on Big Data and Smart Computing, BigComp 2019 - Proceedings. Institute of Electrical and Electronics Engineers Inc. 2019. 8679377. (2019 IEEE International Conference on Big Data and Smart Computing, BigComp 2019 - Proceedings). https://doi.org/10.1109/BIGCOMP.2019.8679377