2.3k citations · 4.7k across the 64 of their papers we have counts for
67 papers
MoE-Lightning: High-Throughput MoE Inference on Memory-constrained GPUs
Shiyi Cao, Shu Liu, Tyler Griggs +6
Efficient deployment of large language models, particularly Mixture of Experts (MoE), on resource-constrained platforms presents significant challenges, especially in terms of comp…
RAFT: Adapting Language Model to Domain Specific RAG
Tianjun Zhang, Shishir G. Patil, Naman Jain +4
Pretraining Large Language Models (LLMs) on large corpora of textual data is now a standard paradigm. When using these LLMs for many downstream applications, it is common to additi…
Optimizing LLM Queries in Relational Data Analytics Workloads
Shu Liu, Asim Biswal, Amog Kamsetty +8
Batch data analytics is a growing application for Large Language Models (LLMs). LLMs enable users to perform a wide range of natural language tasks, such as classification, entity…
Accelerating Aggregation Queries on Unstructured Streams of Data
Matthew Russo, Tatsunori Hashimoto, Daniel Kang +2
Analysts and scientists are interested in querying streams of video, audio, and text to extract quantitative insights. For example, an urban planner may wish to measure congestion…
FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance
Lingjiao Chen, Matei Zaharia, James Zou
There is a rapidly growing number of large language models (LLMs) that users can query for a fee. We review the cost associated with querying popular LLM APIs, e.g. GPT-4, ChatGPT,…
Zelda: Video Analytics using Vision-Language Models
Francisco Romero, Caleb Winston, Johann Hauswald +2
Advances in ML have motivated the design of video analytics systems that allow for structured queries over video datasets. However, existing systems limit query expressivity, requi…
Exploiting Programmatic Behavior of LLMs: Dual-Use Through Standard Security Attacks
Daniel Kang, Xuechen Li, Ion Stoica +3
Recent advances in instruction-following large language models (LLMs) have led to dramatic improvements in a range of NLP tasks. Unfortunately, we find that the same improved capab…
Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLP
Omar Khattab, Keshav Santhanam, Xiang Lisa Li +4
Retrieval-augmented in-context learning has emerged as a powerful approach for addressing knowledge-intensive tasks using frozen language models (LM) and retrieval models (RM). Exi…
Transactions Make Debugging Easy
Qian Li, Peter Kraft, Michael Cafarella +6
We propose TROD, a novel transaction-oriented framework for debugging modern distributed web applications and online services. Our critical insight is that if applications store al…
Moving Beyond Downstream Task Accuracy for Information Retrieval Benchmarking
Keshav Santhanam, Jon Saad-Falcon, Martin Franz +7
Neural information retrieval (IR) systems have progressed rapidly in recent years, in large part due to the release of publicly available benchmarking tasks. Unfortunately, some di…
MegaBlocks: Efficient Sparse Training with Mixture-of-Experts
Trevor Gale, Deepak Narayanan, Cliff Young +1
We present MegaBlocks, a system for efficient Mixture-of-Experts (MoE) training on GPUs. Our system is motivated by the limitations of current frameworks, which restrict the dynami…
HAPI: A Large-scale Longitudinal Dataset of Commercial ML API Predictions
Lingjiao Chen, Zhihua Jin, Sabri Eyuboglu +3
Commercial ML APIs offered by providers such as Google, Amazon and Microsoft have dramatically simplified ML adoption in many applications. Numerous companies and academics pay to…
Estimating and Explaining Model Performance When Both Covariates and Labels Shift
Lingjiao Chen, Matei Zaharia, James Zou
Deployed machine learning (ML) models often encounter new user data that differs from their training data. Therefore, estimating how well a given model might perform on the new dat…
Apiary: A DBMS-Integrated Transactional Function-as-a-Service Framework
Peter Kraft, Qian Li, Kostis Kaffes +16
Developers increasingly use function-as-a-service (FaaS) platforms for data-centric applications that perform low-latency and transactional operations on data, such as for microser…
PLAID: An Efficient Engine for Late Interaction Retrieval
Keshav Santhanam, Omar Khattab, Christopher Potts +1
Pre-trained language models are increasingly important components across multiple information retrieval (IR) paradigms. Late interaction, introduced with the ColBERT model and rece…
Extricating IoT Devices from Vendor Infrastructure with Karl
Gina Yuan, David Mazières, Matei Zaharia
Most consumer IoT devices are vertically integrated with cloud-side infrastructure. Such architectures present enormous risk to user data, exacerbated by vendor heterogeneity and t…
Finding Label and Model Errors in Perception Data With Learned Observation Assertions
Daniel Kang, Nikos Arechiga, Sudeep Pillai +2
ML is being deployed in complex, real-world scenarios where errors have impactful consequences. In these systems, thorough testing of the ML pipelines is critical. A key component…
What can Data-Centric AI Learn from Data and ML Engineering?
Neoklis Polyzotis, Matei Zaharia
Data-centric AI is a new and exciting research topic in the AI community, but many organizations already build and maintain various "data-centric" applications whose goal is to pro…
ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction
Keshav Santhanam, Omar Khattab, Jon Saad-Falcon +2
Neural information retrieval (IR) has greatly advanced search and other knowledge-intensive language tasks. While many neural IR methods encode queries and documents into single-ve…
Toward Compact Parameter Representations for Architecture-Agnostic Neural Network Compression
Yuezhou Sun, Wenlong Zhao, Lijun Zhang +3
This paper investigates deep neural network (DNN) compression from the perspective of compactly representing and storing trained parameters. We explore the previously overlooked op…
DistIR: An Intermediate Representation and Simulator for Efficient Neural Network Distribution
Keshav Santhanam, Siddharth Krishna, Ryota Tomioka +2
The rapidly growing size of deep neural network (DNN) models and datasets has given rise to a variety of distribution strategies such as data, tensor-model, pipeline parallelism, a…
Solving Large-Scale Granular Resource Allocation Problems Efficiently with POP
Deepak Narayanan, Fiodar Kazhamiaka, Firas Abuzaid +5
Resource allocation problems in many computer systems can be formulated as mathematical optimization problems. However, finding exact solutions to these problems using off-the-shel…
Hindsight: Posterior-guided training of retrievers for improved open-ended generation
Ashwin Paranjape, Omar Khattab, Christopher Potts +2
Many text generation systems benefit from using a retriever to retrieve passages from a textual knowledge corpus (e.g., Wikipedia) which are then provided as additional context to…
On the Opportunities and Risks of Foundation Models
Rishi Bommasani, Drew A. Hudson, Ehsan Adeli +111
AI is undergoing a paradigm shift with the rise of models (e.g., BERT, DALL-E, GPT-3) that are trained on broad data at scale and are adaptable to a wide range of downstream tasks.…