5 papers · 1 filter
EinDecomp: Decomposition of Declaratively-Specified Machine Learning and Numerical Computations for Parallel Execution
Daniel Bourgeois, Zhimin Ding, Dimitrije Jankov +6
We consider the problem of automatically decomposing operations over tensors or arrays so that they can be executed in parallel on multiple devices. We address two, closely-linked…
RAG-Modulo: Solving Sequential Tasks using Experience, Critics, and Language Models
Abhinav Jain, Chris Jermaine, Vaibhav Unhelkar
Large language models (LLMs) have recently emerged as promising tools for solving challenging robotic tasks, even in the presence of action and observation uncertainties. Recent LL…
TURNIP: A "Nondeterministic" GPU Runtime with CPU RAM Offload
Zhimin Ding, Jiawen Yao, Brianna Barrow +7
An obvious way to alleviate memory difficulties in GPU-based AI computing is via CPU offload, where data are moved between GPU and CPU RAM, so inexpensive CPU RAM is used to increa…
Prompt Tuning Strikes Back: Customizing Foundation Models with Low-Rank Prompt Adaptation
Abhinav Jain, Swarat Chaudhuri, Thomas Reps +1
Parameter-Efficient Fine-Tuning (PEFT) has become the standard for customising Foundation Models (FMs) to user-specific downstream tasks. However, typical PEFT methods require stor…
Online Cascade Learning for Efficient Inference over Streams
Lunyiu Nie, Zhimin Ding, Erdong Hu +2
Large Language Models (LLMs) have a natural role in answering complex queries about data streams, but the high computational cost of LLM inference makes them infeasible in many suc…