7 papers
LLM Evaluation is a Forecasting Problem
Rahul Thomas, Adam Block, Micah Goldblum
Standard procedure in machine learning is to randomly split train and test data, emulating IID draws from one underlying data distribution. In practice, models are trained on past…
Dynamic Delayed Tree Expansion For Improved Multi-Path Speculative Decoding
Rahul Thomas, Teo Kitanovski, Micah Goldblum +1
Multi-path speculative decoding accelerates lossless sampling from a target model by using a cheaper draft model to generate a draft tree of tokens, and then applies a verification…
Greedy Multi-Path Block Verification for Faster Decoding in Speculative Sampling
Rahul Thomas, Arka Pal, David Hidary
The goal of -step speculative decoding is to accelerate autoregressive decoding of a target model by using a cheaper draft model to generate a candidate path of tokens. Base…
Global Resolution: Optimal Multi-Draft Speculative Sampling via Convex Minimization
Rahul Krishna Thomas, Arka Pal
Speculative sampling reduces the latency of autoregressive decoding for target model LLMs without sacrificing inference quality, by using a cheap draft model to suggest a candidate…
Cascade: Token-Sharded Private LLM Inference
Rahul Thomas, Louai Zahran, Erica Choi +3
As LLMs continue to increase in parameter size, the computational resources required to run them are available to fewer parties. Therefore, third-party inference services -- where…
An Attack to Break Permutation-Based Private Third-Party Inference Schemes for LLMs
Rahul Thomas, Louai Zahran, Erica Choi +3
Recent advances in Large Language Models (LLMs) have led to the widespread adoption of third-party inference services, raising critical privacy concerns. Existing methods of perfor…
Actionable Insights on Philadelphia Crime Hot-Spots: Clustering and Statistical Analysis to Inform Future Crime Legislation
Ishan S. Khare, Tarun K. Martheswaran, Rahul K. Thomas +1
Philadelphia's problem with high crime rates continues to be exacerbated as Philadelphia's residents, community leaders, and law enforcement officials struggle to address the root…