3 citations · 7 across the 10 of their papers we have counts for
10 papers
Design Space Exploration of DMA based Finer-Grain Compute Communication Overlap
Shagnik Pal, Shaizeen Aga, Suchita Pati +2
Modern ML workloads demand distributing training and inference across multiple GPUs. However, these parallelization techniques often suffer from exposed critical-path communication…
DMA-Latte: Expanding the Reach of DMA Offloads to Latency-bound ML Communication
Suchita Pati, Shaizeen Aga, Mahzabeen Islam +3
Offloading communication to existing direct memory access (DMA) engines, available on most state-of-the-art commercial GPUs, has emerged as a practical and low-cost solution to eff…
Optimizing ML Concurrent Computation and Communication with GPU DMA Engines
Anirudha Agrawal, Shaizeen Aga, Suchita Pati +1
Concurrent computation and communication (C3) is a pervasive paradigm in ML and other domains, making its performance optimization crucial. In this paper, we carefully characterize…
Global Optimizations & Lightweight Dynamic Logic for Concurrency
Suchita Pati, Shaizeen Aga, Nuwan Jayasena +1
Modern accelerators like GPUs are increasingly executing independent operations concurrently to improve the device's compute utilization. However, effectively harnessing it on GPUs…
T3: Transparent Tracking & Triggering for Fine-grained Overlap of Compute & Collectives
Suchita Pati, Shaizeen Aga, Mahzabeen Islam +2
Large Language Models increasingly rely on distributed techniques for their training and inference. These techniques require communication across devices which can reduce scaling e…
Just-in-time Quantization with Processing-In-Memory for Efficient ML Training
Mohamed Assem Ibrahim, Shaizeen Aga, Ada Li +2
Data format innovations have been critical for machine learning (ML) scaling, which in turn fuels ground-breaking ML capabilities. However, even in the presence of low-precision fo…
Computation vs. Communication Scaling for Future Transformers on Future Hardware
Suchita Pati, Shaizeen Aga, Mahzabeen Islam +2
Scaling neural network models has delivered dramatic quality gains across ML problems. However, this scaling has increased the reliance on efficient distributed training techniques…
Demystifying BERT: Implications for Accelerator Design
Suchita Pati, Shaizeen Aga, Nuwan Jayasena +1
Transfer learning in natural language processing (NLP), as realized using models like BERT (Bi-directional Encoder Representation from Transformer), has significantly improved lang…
SeqPoint: Identifying Representative Iterations of Sequence-based Neural Networks
Suchita Pati, Shaizeen Aga, Matthew D. Sinclair +1
The ubiquity of deep neural networks (DNNs) continues to rise, making them a crucial application class for hardware optimizations. However, detailed profiling and characterization…
Analyzing Machine Learning Workloads Using a Detailed GPU Simulator
Jonathan Lew, Deval Shah, Suchita Pati +8
Most deep neural networks deployed today are trained using GPUs via high-level frameworks such as TensorFlow and PyTorch. This paper describes changes we made to the GPGPU-Sim simu…