3 citations · 7 across the 10 of their papers we have counts for
6 papers · 1 filter
Optimizing ML Concurrent Computation and Communication with GPU DMA Engines
Anirudha Agrawal, Shaizeen Aga, Suchita Pati +1
Concurrent computation and communication (C3) is a pervasive paradigm in ML and other domains, making its performance optimization crucial. In this paper, we carefully characterize…
Global Optimizations & Lightweight Dynamic Logic for Concurrency
Suchita Pati, Shaizeen Aga, Nuwan Jayasena +1
Modern accelerators like GPUs are increasingly executing independent operations concurrently to improve the device's compute utilization. However, effectively harnessing it on GPUs…
T3: Transparent Tracking & Triggering for Fine-grained Overlap of Compute & Collectives
Suchita Pati, Shaizeen Aga, Mahzabeen Islam +2
Large Language Models increasingly rely on distributed techniques for their training and inference. These techniques require communication across devices which can reduce scaling e…
Just-in-time Quantization with Processing-In-Memory for Efficient ML Training
Mohamed Assem Ibrahim, Shaizeen Aga, Ada Li +2
Data format innovations have been critical for machine learning (ML) scaling, which in turn fuels ground-breaking ML capabilities. However, even in the presence of low-precision fo…
Computation vs. Communication Scaling for Future Transformers on Future Hardware
Suchita Pati, Shaizeen Aga, Mahzabeen Islam +2
Scaling neural network models has delivered dramatic quality gains across ML problems. However, this scaling has increased the reliance on efficient distributed training techniques…
Demystifying BERT: Implications for Accelerator Design
Suchita Pati, Shaizeen Aga, Nuwan Jayasena +1
Transfer learning in natural language processing (NLP), as realized using models like BERT (Bi-directional Encoder Representation from Transformer), has significantly improved lang…