17 citations · 41 across the 11 of their papers we have counts for
13 papers
GPU Goldmines: Tunable WebGPU Kernels for Edge LLM Inference
Rithik Sharma, Reese Levine, Zheyuan Chen +3
As demand for LLM inference grows and consumer GPUs grow more capable, edge deployment is becoming increasingly attractive. WebGPU is compelling in this setting because it offers c…
WarpDRF: The Unwritten Contract of GPU Warp Programming
Zheyuan Chen, Simon Kagle, Tiago Cogumbreiro +1
Warp primitives such as tensor core operations, shuffles, reductions, and barriers are critical to high-performance GPU kernels, and every major GPU language supports some set of t…
Tile-Level Activation Overlap for Efficient LLM Inference
Abhinav Jangda, Tyler Sorensen, Sebastian Burckhardt +3
SwiGLU is the dominant MLP activation in modern large language models, yet its intermediate tensor materialization costs 9-37% of MLP execution time. We present two complementary C…
Llamas on the Web: Memory-Efficient, Performance-Portable, and Multi-Precision LLM Inference with WebGPU
Reese Levine, Rithik Sharma, Nikhil Jain +5
Running language models in the browser presents a unique opportunity to build efficient, private, and portable AI applications, but requires contending with constrained memory avai…
Memory DisOrder: Memory Re-orderings as a Timerless Side-channel
Sean Siddens, Sanya Srivastava, Reese Levine +2
To improve efficiency, nearly all parallel processing units (CPUs and GPUs) implement relaxed memory models in which memory operations may be re-ordered, i.e., executed out-of-orde…
PEAK: A Performance Engineering AI-Assistant for GPU Kernels Powered by Natural Language Transformations
Muhammad Usman Tariq, Abhinav Jangda, Angelica Moreira +2
Advancements in large language models (LLMs) are showing promising impact in software development and programming assistance. However, these models struggle when operating on low-l…
Mix Testing: Specifying and Testing ABI Compatibility of C/C++ Atomics Implementations
Luke Geeson, James Brotherston, Wilco Dijkstra +4
The correctness of complex software depends on the correctness of both the source code and the compilers that generate corresponding binary code. Compilers must do more than preser…
LeftoverLocals: Listening to LLM Responses Through Leaked GPU Local Memory
Tyler Sorensen, Heidy Khlaaf
This paper describes LeftoverLocals: a vulnerability that allows data recovery from GPU memory created by another process on Apple, Qualcomm, and AMD GPUs. LeftoverLocals impacts t…
Specifying and Testing GPU Workgroup Progress Models
Tyler Sorensen, Lucas F. Salvador, Harmit Raval +4
As GPU availability has increased and programming support has matured, a wider variety of applications are being ported to these platforms. Many parallel applications contain fine-…
The MosaicSim Simulator (Full Technical Report)
Opeoluwa Matthews, Aninda Manocha, Davide Giri +7
As Moore's Law has slowed and Dennard Scaling has ended, architects are increasingly turning to heterogeneous parallelism and domain-specific hardware-software co-designs. These tr…
Do Your Cores Play Nicely? A Portable Framework for Multi-core Interference Tuning and Analysis
Dan Iorga, Tyler Sorensen, Alastair F. Donaldson
Multi-core architectures can be leveraged to allow independent processes to run in parallel. However, due to resources shared across cores, such as caches, distinct processes may i…
The Semantics of Transactions and Weak Memory in x86, Power, ARM, and C++
Nathan Chong, Tyler Sorensen, John Wickerson
Weak memory models provide a complex, system-centric semantics for concurrent programs, while transactional memory (TM) provides a simpler, programmer-centric semantics. Both have…
Cooperative Kernels: GPU Multitasking for Blocking Algorithms (Extended Version)
Tyler Sorensen, Hugues Evrard, Alastair F. Donaldson
There is growing interest in accelerating irregular data-parallel algorithms on GPUs. These algorithms are typically blocking, so they require fair scheduling. But GPU programming…