Asynchronous I/O stack: A Low-latency Kernel I/O Stack for Ultra-Low Latency SSDs

Gyusun Lee; Seokha Shin; Wonsuk Song; Tae Jun Ham; Jae W. Lee; Jinkyu Jeong

Asynchronous I/O stack: A Low-latency Kernel I/O Stack for Ultra-Low Latency SSDs

Gyusun Lee, Seokha Shin, Wonsuk Song, Tae Jun Ham, Jae W. Lee, Jinkyu Jeong

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Today’s ultra-low latency SSDs can deliver an I/O latency of sub-ten microseconds. With this dramatically shrunken device time, operations inside the kernel I/O stack, which were traditionally considered lightweight, are no longer a negligible portion. This motivates us to reexamine the storage I/O stack design and propose an asynchronous I/O stack (AIOS), where synchronous operations in the I/O path are replaced by asynchronous ones to overlap I/O-related CPU operations with device I/O. The asynchronous I/O stack leverages a lightweight block layer specialized for NVMe SSDs using the page cache without block I/O scheduling and merging, thereby reducing the sojourn time in the block layer. We prototype the proposed asynchronous I/O stack on the Linux kernel and evaluate it with various workloads. Synthetic FIO benchmarks demonstrate that the application-perceived I/O latency falls into single-digit microseconds for 4 KB random reads on Optane SSD, and the overall I/O latency is reduced by 15–33% across varying block sizes. This I/O latency reduction leads to a significant performance improvement of real-world applications as well: 11–44% IOPS increase on RocksDB and 15–30% throughput improvement on Filebench and OLTP workloads.

Original language	English
Title of host publication	Proceedings of the 2019 USENIX Annual Technical Conference, USENIX ATC 2019
Publisher	USENIX Association
Pages	603-616
Number of pages	14
ISBN (Electronic)	9781939133038
Publication status	Published - 2019
Event	2019 USENIX Annual Technical Conference, USENIX ATC 2019 - Renton, United States Duration: 2019 Jul 10 → 2019 Jul 12

Publication series

Name	Proceedings of the 2019 USENIX Annual Technical Conference, USENIX ATC 2019

Conference

Conference	2019 USENIX Annual Technical Conference, USENIX ATC 2019
Country/Territory	United States
City	Renton
Period	19/7/10 → 19/7/12

Bibliographical note

Funding Information:
We would like to thank the anonymous reviewers and our shepherd, Youjip Won, for their valuable comments. We also thank Prof. Jin-Soo Kim for his devotion of time at LAX and valuable technical feedback. This work was supported partly by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (NRF-2017R1C1B2007273, NRF-2016M3C4A7952587) and by Samsung Electronics.

Publisher Copyright:
© Proceedings of the 2019 USENIX Annual Technical Conference, USENIX ATC 2019. All rights reserved.

All Science Journal Classification (ASJC) codes

Computer Science(all)

Cite this

@inproceedings{5bed33d9cb4641dd957564b16b1cd254,

title = "Asynchronous I/O stack: A Low-latency Kernel I/O Stack for Ultra-Low Latency SSDs",

abstract = "Today{\textquoteright}s ultra-low latency SSDs can deliver an I/O latency of sub-ten microseconds. With this dramatically shrunken device time, operations inside the kernel I/O stack, which were traditionally considered lightweight, are no longer a negligible portion. This motivates us to reexamine the storage I/O stack design and propose an asynchronous I/O stack (AIOS), where synchronous operations in the I/O path are replaced by asynchronous ones to overlap I/O-related CPU operations with device I/O. The asynchronous I/O stack leverages a lightweight block layer specialized for NVMe SSDs using the page cache without block I/O scheduling and merging, thereby reducing the sojourn time in the block layer. We prototype the proposed asynchronous I/O stack on the Linux kernel and evaluate it with various workloads. Synthetic FIO benchmarks demonstrate that the application-perceived I/O latency falls into single-digit microseconds for 4 KB random reads on Optane SSD, and the overall I/O latency is reduced by 15–33% across varying block sizes. This I/O latency reduction leads to a significant performance improvement of real-world applications as well: 11–44% IOPS increase on RocksDB and 15–30% throughput improvement on Filebench and OLTP workloads.",

author = "Gyusun Lee and Seokha Shin and Wonsuk Song and Ham, {Tae Jun} and Lee, {Jae W.} and Jinkyu Jeong",

note = "Funding Information: We would like to thank the anonymous reviewers and our shepherd, Youjip Won, for their valuable comments. We also thank Prof. Jin-Soo Kim for his devotion of time at LAX and valuable technical feedback. This work was supported partly by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (NRF-2017R1C1B2007273, NRF-2016M3C4A7952587) and by Samsung Electronics. Publisher Copyright: {\textcopyright} Proceedings of the 2019 USENIX Annual Technical Conference, USENIX ATC 2019. All rights reserved.; 2019 USENIX Annual Technical Conference, USENIX ATC 2019 ; Conference date: 10-07-2019 Through 12-07-2019",

year = "2019",

language = "English",

series = "Proceedings of the 2019 USENIX Annual Technical Conference, USENIX ATC 2019",

publisher = "USENIX Association",

pages = "603--616",

booktitle = "Proceedings of the 2019 USENIX Annual Technical Conference, USENIX ATC 2019",

}

Lee, G, Shin, S, Song, W, Ham, TJ, Lee, JW & Jeong, J 2019, Asynchronous I/O stack: A Low-latency Kernel I/O Stack for Ultra-Low Latency SSDs. in Proceedings of the 2019 USENIX Annual Technical Conference, USENIX ATC 2019. Proceedings of the 2019 USENIX Annual Technical Conference, USENIX ATC 2019, USENIX Association, pp. 603-616, 2019 USENIX Annual Technical Conference, USENIX ATC 2019, Renton, United States, 19/7/10.

Asynchronous I/O stack : A Low-latency Kernel I/O Stack for Ultra-Low Latency SSDs. / Lee, Gyusun; Shin, Seokha; Song, Wonsuk et al.

Proceedings of the 2019 USENIX Annual Technical Conference, USENIX ATC 2019. USENIX Association, 2019. p. 603-616 (Proceedings of the 2019 USENIX Annual Technical Conference, USENIX ATC 2019).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Asynchronous I/O stack

T2 - 2019 USENIX Annual Technical Conference, USENIX ATC 2019

AU - Lee, Gyusun

AU - Shin, Seokha

AU - Song, Wonsuk

AU - Ham, Tae Jun

AU - Lee, Jae W.

AU - Jeong, Jinkyu

N1 - Funding Information: We would like to thank the anonymous reviewers and our shepherd, Youjip Won, for their valuable comments. We also thank Prof. Jin-Soo Kim for his devotion of time at LAX and valuable technical feedback. This work was supported partly by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (NRF-2017R1C1B2007273, NRF-2016M3C4A7952587) and by Samsung Electronics. Publisher Copyright: © Proceedings of the 2019 USENIX Annual Technical Conference, USENIX ATC 2019. All rights reserved.

PY - 2019

Y1 - 2019

N2 - Today’s ultra-low latency SSDs can deliver an I/O latency of sub-ten microseconds. With this dramatically shrunken device time, operations inside the kernel I/O stack, which were traditionally considered lightweight, are no longer a negligible portion. This motivates us to reexamine the storage I/O stack design and propose an asynchronous I/O stack (AIOS), where synchronous operations in the I/O path are replaced by asynchronous ones to overlap I/O-related CPU operations with device I/O. The asynchronous I/O stack leverages a lightweight block layer specialized for NVMe SSDs using the page cache without block I/O scheduling and merging, thereby reducing the sojourn time in the block layer. We prototype the proposed asynchronous I/O stack on the Linux kernel and evaluate it with various workloads. Synthetic FIO benchmarks demonstrate that the application-perceived I/O latency falls into single-digit microseconds for 4 KB random reads on Optane SSD, and the overall I/O latency is reduced by 15–33% across varying block sizes. This I/O latency reduction leads to a significant performance improvement of real-world applications as well: 11–44% IOPS increase on RocksDB and 15–30% throughput improvement on Filebench and OLTP workloads.

AB - Today’s ultra-low latency SSDs can deliver an I/O latency of sub-ten microseconds. With this dramatically shrunken device time, operations inside the kernel I/O stack, which were traditionally considered lightweight, are no longer a negligible portion. This motivates us to reexamine the storage I/O stack design and propose an asynchronous I/O stack (AIOS), where synchronous operations in the I/O path are replaced by asynchronous ones to overlap I/O-related CPU operations with device I/O. The asynchronous I/O stack leverages a lightweight block layer specialized for NVMe SSDs using the page cache without block I/O scheduling and merging, thereby reducing the sojourn time in the block layer. We prototype the proposed asynchronous I/O stack on the Linux kernel and evaluate it with various workloads. Synthetic FIO benchmarks demonstrate that the application-perceived I/O latency falls into single-digit microseconds for 4 KB random reads on Optane SSD, and the overall I/O latency is reduced by 15–33% across varying block sizes. This I/O latency reduction leads to a significant performance improvement of real-world applications as well: 11–44% IOPS increase on RocksDB and 15–30% throughput improvement on Filebench and OLTP workloads.

UR - http://www.scopus.com/inward/record.url?scp=85077061531&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85077061531&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:85077061531

T3 - Proceedings of the 2019 USENIX Annual Technical Conference, USENIX ATC 2019

SP - 603

EP - 616

BT - Proceedings of the 2019 USENIX Annual Technical Conference, USENIX ATC 2019

PB - USENIX Association

Y2 - 10 July 2019 through 12 July 2019

ER -

Asynchronous I/O stack: A Low-latency Kernel I/O Stack for Ultra-Low Latency SSDs

Abstract

Publication series

Conference

Bibliographical note

All Science Journal Classification (ASJC) codes

Other files and links

Fingerprint

Cite this