3 papers
cs.CL2025
Training Domain Draft Models for Speculative Decoding: Best Practices and Insights
Fenglu Hong, Ravi Raju, Jonathan Lingjie Li +5
Speculative decoding is an effective method for accelerating inference of large language models (LLMs) by employing a small draft model to predict the output of a target model. How…
cs.LG2024
Composition of Experts: A Modular Compound AI System Leveraging Large Language Models
Swayambhoo Jain, Ravi Raju, Bo Li +8
Large Language Models (LLMs) have achieved remarkable advancements, but their monolithic nature presents challenges in terms of scalability, cost, and customization. This paper int…
cs.AR2024
SambaNova SN40L: Scaling the AI Memory Wall with Dataflow and Composition of Experts
Raghu Prabhakar, Ram Sivaramakrishnan, Darshan Gandhi +27
Monolithic large language models (LLMs) like GPT-4 have paved the way for modern generative AI applications. Training, serving, and maintaining monolithic LLMs at scale, however, r…