4 citations · 20 across the 17 of their papers we have counts for
3 papers · 2 filters
QUASAR: Lowering the Loss Floor of Quantization-Aware Training with Loss-Aware Reconstruction
Vincent Counathe, Ben Athiwaratkun, Christopher De Sa +1
As large language model inference shifts toward lower precision, post-training quantization (PTQ) becomes increasingly brittle, making quantization-aware training (QAT) essential f…
Extractable Memorization From First Principles
A. Feder Cooper, Marika Swanberg, Jamie Hayes +5
Recent work on extractable memorization in LLMs suffers from two contrasting validity problems. Some studies overstate extraction, e.g., relying on sequences too short to distingui…
L: Large Lookup Layers
Albert Tseng, Christopher De Sa
Modern sparse language models typically achieve sparsity through Mixture-of-Experts (MoE) layers, which dynamically route tokens to dense MLP "experts." However, dynamic hard routi…