17 citations · 41 across the 11 of their papers we have counts for
4 papers · 1 filter
GPU Goldmines: Tunable WebGPU Kernels for Edge LLM Inference
Rithik Sharma, Reese Levine, Zheyuan Chen +3
As demand for LLM inference grows and consumer GPUs grow more capable, edge deployment is becoming increasingly attractive. WebGPU is compelling in this setting because it offers c…
Tile-Level Activation Overlap for Efficient LLM Inference
Abhinav Jangda, Tyler Sorensen, Sebastian Burckhardt +3
SwiGLU is the dominant MLP activation in modern large language models, yet its intermediate tensor materialization costs 9-37% of MLP execution time. We present two complementary C…
Llamas on the Web: Memory-Efficient, Performance-Portable, and Multi-Precision LLM Inference with WebGPU
Reese Levine, Rithik Sharma, Nikhil Jain +5
Running language models in the browser presents a unique opportunity to build efficient, private, and portable AI applications, but requires contending with constrained memory avai…
Do Your Cores Play Nicely? A Portable Framework for Multi-core Interference Tuning and Analysis
Dan Iorga, Tyler Sorensen, Alastair F. Donaldson
Multi-core architectures can be leveraged to allow independent processes to run in parallel. However, due to resources shared across cores, such as caches, distinct processes may i…