171 citations · 729 across the 42 of their papers we have counts for
42 papers
Networks of Networks: Complexity Class Principles Applied to Compound AI Systems Design
Jared Quincy Davis, Boris Hanin, Lingjiao Chen +3
As practitioners seek to surpass the current reliability and quality frontier of monolithic models, Compound AI Systems consisting of many language model inference calls are increa…
Are More LLM Calls All You Need? Towards Scaling Laws of Compound Inference Systems
Lingjiao Chen, Jared Quincy Davis, Boris Hanin +4
Many recent state-of-the-art results in language tasks were achieved using compound systems that perform multiple Language Model (LM) calls and aggregate their responses. However,…
Break the Sequential Dependency of LLM Inference Using Lookahead Decoding
Yichao Fu, Peter Bailis, Ion Stoica +1
Autoregressive decoding of large language models (LLMs) is memory bandwidth bounded, resulting in high latency and significant wastes of the parallel processing power of modern acc…
Online Speculative Decoding
Xiaoxuan Liu, Lanxiang Hu, Peter Bailis +4
Speculative decoding is a pivotal technique to accelerate the inference of large language models (LLMs) by employing a smaller draft model to predict the target model's outputs. Ho…
Apiary: A DBMS-Integrated Transactional Function-as-a-Service Framework
Peter Kraft, Qian Li, Kostis Kaffes +16
Developers increasingly use function-as-a-service (FaaS) platforms for data-centric applications that perform low-latency and transactional operations on data, such as for microser…
Finding Label and Model Errors in Perception Data With Learned Observation Assertions
Daniel Kang, Nikos Arechiga, Sudeep Pillai +2
ML is being deployed in complex, real-world scenarios where errors have impactful consequences. In these systems, thorough testing of the ML pipelines is critical. A key component…