113 citations · 191 across the 22 of their papers we have counts for
4 papers · 1 filter
Network and Compiler Optimizations for Efficient Linear Algebra Kernels in Private Transformer Inference
Karthik Garimella, Negar Neda, Austin Ebel +2
Large language model (LLM) based services are primarily structured as client-server interactions, with clients sending queries directly to cloud providers that host LLMs. This appr…
Spectral Scaling Laws in Language Models: How Effectively Do Feed-Forward Networks Use Their Latent Space?
Nandan Kumar Jha, Brandon Reagen
As large language models (LLMs) scale, the question is not only how large they become, but how much of their capacity is effectively utilized. Existing scaling laws relate model si…
A Random Matrix Theory Perspective on the Learning Dynamics of Multi-head Latent Attention
Nandan Kumar Jha, Brandon Reagen
In this work, we study how multi-head latent attention (MLA), a popular strategy for compressing key/value memory, affects a transformer's internal capacity during pretraining. Usi…
Entropy-Guided Attention for Private LLMs
Nandan Kumar Jha, Brandon Reagen
The pervasiveness of proprietary language models has raised critical privacy concerns, necessitating advancements in private inference (PI), where computations are performed direct…