3 papers
cs.AI2026
Refusal-Gated Decoding: Preserving Refusal Behavior Under High-Temperature Sampling
Phillip Howard, Xin Su, Allen Roush +2
High-temperature sampling is one of the primary mechanisms for increasing diversity in LLMs. Recent advances in truncation-based sampling techniques have helped mitigate drawbacks…
cs.LG2026
Spectral Superposition: A Theory of Feature Geometry
Georgi Ivanov, Narmeen Oozeer, Shivam Raval +3
Neural networks represent more features than they have dimensions via superposition, forcing features to share representational space. Current methods decompose activations into sp…
cs.CY2025
Beyond Monoliths: Expert Orchestration for More Capable, Democratic, and Safe Language Models
Philip Quirke, Narmeen Oozeer, Chaithanya Bandi +8
This position paper argues that the prevailing trajectory toward ever larger, more expensive generalist foundation models controlled by a handful of companies limits innovation and…