20 citations · 29 across the 10 of their papers we have counts for
4 papers · 1 filter
Pinocchio: Fast Uncertainty Estimates for Black-Box Language Models
Kevin David Hayes, Arka Pal, Haosong Zhang +2
In high-stakes decision-making applications of large language models (LLMs), practitioners require not only accurate LLMs but also uncertainty estimates for their predictions. Exis…
Privacy-Preserving Mechanisms Enable Cheap Verifiable Inference of LLMs
Arka Pal, Louai Zahran, William Gvozdjak +2
As large language models (LLMs) continue to grow in size, fewer users are able to host and run models locally. This has led to increased use of third-party hosting services. Howeve…
Dynamic Delayed Tree Expansion For Improved Multi-Path Speculative Decoding
Rahul Thomas, Teo Kitanovski, Micah Goldblum +1
Multi-path speculative decoding accelerates lossless sampling from a target model by using a cheaper draft model to generate a draft tree of tokens, and then applies a verification…
Greedy Multi-Path Block Verification for Faster Decoding in Speculative Sampling
Rahul Thomas, Arka Pal, David Hidary
The goal of -step speculative decoding is to accelerate autoregressive decoding of a target model by using a cheaper draft model to generate a candidate path of tokens. Base…