Exploring Modern GPU Memory System Design Challenges through Accurate Modeling

Abstract

This paper explores the impact of simulator accuracy on architecture design decisions in the general-purpose graphics processing unit (GPGPU) space. We perform a detailed, quantitative analysis of the most popular publicly available GPU simulator, GPGPU-Sim, against our enhanced version of the simulator, updated to model the memory system of modern GPUs in more detail. Our enhanced GPU model is able to describe the NVIDIA Volta architecture in sufficient detail to reduce error in memory system even counters by as much as 66X. The reduced error in the memory system further reduces execution time error versus real hardware by 2.5X. To demonstrate the accuracy of our enhanced model against a real machine, we perform a counter-by-counter validation against an NVIDIA TITAN V Volta GPU, demonstrating the relative accuracy of the new simulator versus the publicly available model. We go on to demonstrate that the simpler model discounts the importance of advanced memory system designs such as out-of-order memory access scheduling, while overstating the impact of more heavily researched areas like L1 cache bypassing. Our results demonstrate that it is important for the academic community to enhance the level of detail in architecture simulators as system complexity continues to grow. As part of this detailed correlation and modeling effort, we developed a new Correlator toolset that includes a consolidation of applications from a variety of popular GPGPU benchmark suites, designed to run in reasonable simulation times. The Correlator also includes a database of hardware profiling results for all these applications on NVIDIA cards ranging from Fermi to Volta and a toolchain that enables users to gather correlation statistics and create detailed counter-by-counter hardware correlation plots with minimal effort.

Cited by in corpus (10)

• OliVe: Accelerating Large Language Models via Hardware-friendly Outlier-Victim Pair Quantization
• Bandwidth-Effective DRAM Cache for GPUs with Storage-Class Memory
• Field-Programmable Gate Array Architecture for Deep Learning: Survey & Future Directions
• LUT Tensor Core: A Software-Hardware Co-Design for LUT-Based Low-Bit LLM Inference
• Accelerating Sparse Approximate Matrix Multiplication on GPUs
• Taming Offload Overheads in a Massively Parallel Open-Source RISC-V MPSoC: Analysis and Optimization
• Dual-side Sparse Tensor Core
• Analyzing Modern NVIDIA GPU cores
• Characterizing Concurrency Mechanisms for NVIDIA GPUs under Deep Learning Workloads
• Predictive Data Race Detection for GPUs