5 papers · 1 filter
Dynamic Delayed Tree Expansion For Improved Multi-Path Speculative Decoding
Rahul Thomas, Teo Kitanovski, Micah Goldblum +1
Multi-path speculative decoding accelerates lossless sampling from a target model by using a cheaper draft model to generate a draft tree of tokens, and then applies a verification…
Knowing What You Know Is Not Enough: Large Language Model Confidences Don't Align With Their Actions
Arka Pal, Teo Kitanovski, Arthur Liang +2
Large language models (LLMs) are increasingly deployed in agentic and multi-turn workflows where they are tasked to perform actions of significant consequence. In order to deploy t…
Large Language Models Must Be Taught to Know What They Don't Know
Sanyam Kapoor, Nate Gruver, Manley Roberts +7
When using large language models (LLMs) in high-stakes applications, we need to know when we can trust their predictions. Some works argue that prompting high-performance LLMs is s…
Cascade: Token-Sharded Private LLM Inference
Rahul Thomas, Louai Zahran, Erica Choi +3
As LLMs continue to increase in parameter size, the computational resources required to run them are available to fewer parties. Therefore, third-party inference services -- where…
vTune: Verifiable Fine-Tuning for LLMs Through Backdooring
Eva Zhang, Arka Pal, Akilesh Potti +1
As fine-tuning large language models (LLMs) becomes increasingly prevalent, users often rely on third-party services with limited visibility into their fine-tuning processes. This…