4 papers
Support Tokens, Stability Margins, and a New Foundation for Robust LLMs
Deepak Agarwal, Dhyey Dharmendrakumar Mavani, Suyash Gupta +2
Self-attention is usually described as a flexible, content-adaptive way to mix a token with information from its past. We reinterpret causal self-attention transformers, the backbo…
LLM Query Scheduling with Prefix Reuse and Latency Constraints
Gregory Dexter, Shao Tang, Ata Fatahi Baarzi +3
The efficient deployment of large language models (LLMs) in online settings requires optimizing inference performance under stringent latency constraints, particularly the time-to-…
Scaling Down, Serving Fast: Compressing and Deploying Efficient LLMs for Recommendation Systems
Kayhan Behdin, Ata Fatahibaarzi, Qingquan Song +17
Large language models (LLMs) have demonstrated remarkable performance across a wide range of industrial applications, from search and recommendation systems to generative tasks. Al…
Enhancing Stability for Large Language Models Training in Constrained Bandwidth Networks
Yun Dai, Tejas Dharamsi, Byron Hsu +2
Training extremely large language models (LLMs) with billions of parameters is a computationally intensive task that pushes the limits of current data parallel training systems. Wh…