Access patern-aware cache management for improving data utilization in GPU

Gunjae Koo, Yunho Oh, Won Woo Ro, Murali Annavaram

Research output: Chapter in Book/Report/Conference proceedingConference contribution

18 Citations (Scopus)

Abstract

Long latency of memory operation is a prominent performance bottleneck in graphics processing units (GPUs). The small data cache that must be shared across dozens of warps (a collection of threads) creates signifcant cache contention and premature data eviction. Prior works have recognized this problem and proposed warp throttling which reduces the number of active warps contending for cache space. In this paper we discover that individual load instructions in a warp exhibit four different types of data locality behavior: (1) data brought by a warp load instruction is used only once, which is classifed as streaming data (2) data brought by a warp load is reused multiple times within the same warp, called intra-warp locality (3) data brought by a warp is reused multiple times but across different warps, called inter-warp locality (4) and some data exhibit both a mix of intra-and inter-warp locality. Furthermore, each load instruction exhibits consistently the same locality type across all warps within a GPU kernel. Based on this discovery we argue that cache management must be done using per-load locality type information, rather than applying warp-wide cache management policies. We propose Access Pattern-aware Cache Management (APCM), which dynamically detects the locality type of each load instruction by monitoring the accesses from one exemplary warp. APCM then uses the detected locality type to selectively apply cache bypassing and cache pinning of data based on load locality characterization. Using an extensive set of simulations we show that APCM improves performance of GPUs by 34% for cache sensitive applications while saving 27% of energy consumption over baseline GPU.

Original languageEnglish
Title of host publicationISCA 2017 - 44th Annual International Symposium on Computer Architecture - Conference Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages307-319
Number of pages13
ISBN (Electronic)9781450348928
DOIs
Publication statusPublished - 2017 Jun 24
Event44th Annual International Symposium on Computer Architecture - ISCA 2017 - Toronto, Canada
Duration: 2017 Jun 242017 Jun 28

Publication series

NameProceedings - International Symposium on Computer Architecture
VolumePart F128643
ISSN (Print)1063-6897

Other

Other44th Annual International Symposium on Computer Architecture - ISCA 2017
CountryCanada
CityToronto
Period17/6/2417/6/28

All Science Journal Classification (ASJC) codes

  • Hardware and Architecture

Fingerprint Dive into the research topics of 'Access patern-aware cache management for improving data utilization in GPU'. Together they form a unique fingerprint.

  • Cite this

    Koo, G., Oh, Y., Ro, W. W., & Annavaram, M. (2017). Access patern-aware cache management for improving data utilization in GPU. In ISCA 2017 - 44th Annual International Symposium on Computer Architecture - Conference Proceedings (pp. 307-319). (Proceedings - International Symposium on Computer Architecture; Vol. Part F128643). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1145/3079856.3080239