Pattern analysis based data management method and memory-disk integrated system for high performance computing

This research proposes a memory-disk integrated system (MDIS) with pattern analysis based data management method. The proposed system consists of a pattern adaptive prefetcher with DRAM-based dual buffers and a PCM-based persistent memory structure. The PCM-based persistent memory module is exploited as conventional main memory and storage layers simultaneously. In this system, to compensate the PCMs unfavorable characteristics, a pattern analysis based prefetching method by using dual DRAM buffers with a small amount of space are designed as a sort of caching layer. The proposed pattern analysis based data management method monitors the miss rate to determine the prefetch policy, e.g., when and what to prefetch. These schemes enable flexible adaptation to the application execution characteristics showing unpredictable and inconsistent workload patterns. In order to evaluate our system, we implemented a trace-driven simulator system and launched the YCSB on Redis, Apache Storm, Apache Spark, and OpenStack Swift. The experimental result shows that the MDIS with a pattern adaptive prefetcher can reduce the total access time by 13.9% compared to the conventional scheme and thus the overall performance of the memory-disk integrated system improves.