13 papers
Every Kernel Is a Join: Automatic Multi-GPU Parallelism for AI Computations in Einsummable
Zhimin Ding, Chen-Kuan Liao, Chima Adiole +7
Distributing an AI computation across the GPUs of a multi-GPU server is one of the central problems in systems-for-AI. We present Einsummable, a prototype system that accepts a PyT…
Cost-Aware Optimization for Agentic Query Execution
Lunyiu Nie, Yilin Xia, Yiren Liu +2
Classical query optimization searches over algebraically equivalent plans that differ only in cost. This assumption breaks once LLM-backed operators enter the picture: their placem…
Automated Tensor-Relational Decomposition for Large-Scale Sparse Tensor Computation
Yuxin Tang, Zhiyuan Xin, Zhimin Ding +4
A \emph{tensor-relational} computation is a relational computation where individual tuples carry vectors, matrices, or higher-dimensional arrays. An advantage of tensor-relational…
Auditing Information Disclosure During Large-Scale Gradient-Based Training via Gradient Uniqueness
Sleem Abdelghafar, Maryam Aliakbarpour, Chris Jermaine +1
Auditing information disclosure across every datapoint during the training of LLMs is challenging. We propose a principled, attack-agnostic approach that uses mutual information to…
MHA-RAG: Improving Efficiency, Accuracy, and Consistency by Encoding Exemplars as Soft Prompts
Abhinav Jain, Xinyu Yao, Thomas Reps +1
Adapting Foundation Models to new domains with limited training data is challenging and computationally expensive. While prior work has demonstrated the effectiveness of using doma…
SIMCOPILOT: Evaluating Large Language Models for Copilot-Style Code Generation
Mingchao Jiang, Abhinav Jain, Sophia Zorek +1
We introduce SIMCOPILOT, a benchmark that simulates the role of large language models (LLMs) as interactive, "copilot"-style coding assistants. Targeting both completion (finishing…
DOPPLER: Dual-Policy Learning for Device Assignment in Asynchronous Dataflow Graphs
Xinyu Yao, Daniel Bourgeois, Abhinav Jain +5
We study the problem of assigning operations in a dataflow graph to devices to minimize execution time in a work-conserving system, with emphasis on complex machine learning worklo…
Resource-efficient Inference with Foundation Model Programs
Lunyiu Nie, Zhimin Ding, Kevin Yu +3
The inference-time resource costs of large language and vision models present a growing challenge in production deployments. We propose the use of foundation model programs, i.e.,…
EinDecomp: Decomposition of Declaratively-Specified Machine Learning and Numerical Computations for Parallel Execution
Daniel Bourgeois, Zhimin Ding, Dimitrije Jankov +6
We consider the problem of automatically decomposing operations over tensors or arrays so that they can be executed in parallel on multiple devices. We address two, closely-linked…
RAG-Modulo: Solving Sequential Tasks using Experience, Critics, and Language Models
Abhinav Jain, Chris Jermaine, Vaibhav Unhelkar
Large language models (LLMs) have recently emerged as promising tools for solving challenging robotic tasks, even in the presence of action and observation uncertainties. Recent LL…
TURNIP: A "Nondeterministic" GPU Runtime with CPU RAM Offload
Zhimin Ding, Jiawen Yao, Brianna Barrow +7
An obvious way to alleviate memory difficulties in GPU-based AI computing is via CPU offload, where data are moved between GPU and CPU RAM, so inexpensive CPU RAM is used to increa…
Prompt Tuning Strikes Back: Customizing Foundation Models with Low-Rank Prompt Adaptation
Abhinav Jain, Swarat Chaudhuri, Thomas Reps +1
Parameter-Efficient Fine-Tuning (PEFT) has become the standard for customising Foundation Models (FMs) to user-specific downstream tasks. However, typical PEFT methods require stor…
Online Cascade Learning for Efficient Inference over Streams
Lunyiu Nie, Zhimin Ding, Erdong Hu +2
Large Language Models (LLMs) have a natural role in answering complex queries about data streams, but the high computational cost of LLM inference makes them infeasible in many suc…