4 papers
Llamas on the Web: Memory-Efficient, Performance-Portable, and Multi-Precision LLM Inference with WebGPU
Reese Levine, Rithik Sharma, Nikhil Jain +5
Running language models in the browser presents a unique opportunity to build efficient, private, and portable AI applications, but requires contending with constrained memory avai…
Tile-Level Activation Overlap for Efficient LLM Inference
Abhinav Jangda, Tyler Sorensen, Sebastian Burckhardt +3
SwiGLU is the dominant MLP activation in modern large language models, yet its intermediate tensor materialization costs 9-37% of MLP execution time. We present two complementary C…
Memory DisOrder: Memory Re-orderings as a Timerless Side-channel
Sean Siddens, Sanya Srivastava, Reese Levine +2
To improve efficiency, nearly all parallel processing units (CPUs and GPUs) implement relaxed memory models in which memory operations may be re-ordered, i.e., executed out-of-orde…
PEAK: A Performance Engineering AI-Assistant for GPU Kernels Powered by Natural Language Transformations
Muhammad Usman Tariq, Abhinav Jangda, Angelica Moreira +2
Advancements in large language models (LLMs) are showing promising impact in software development and programming assistance. However, these models struggle when operating on low-l…