55 citations · 106 across the 17 of their papers we have counts for
17 papers
PolyAlign: Conditional Human-Distribution Alignment
L. D. M. S. Sai Teja, Ufaq Khan, Sathira Silva +2
Post-training methods such as supervised fine-tuning (SFT) and preference optimization typically align language models toward a single global assistant behavior. While effective fo…
GRAID: Enhancing Spatial Reasoning of VLMs Through High-Fidelity Data Generation
Karim Elmaaroufi, Liheng Lai, Justin Svegliato +3
Vision Language Models (VLMs) achieve strong performance on many vision-language tasks but often struggle with spatial reasoning$\unicode{x2014}$a prerequisite for many application…
EXP-Bench: Can AI Conduct AI Research Experiments?
Patrick Tser Jern Kon, Jiachen Liu, Xinyi Zhu +10
Automating AI research holds immense potential for accelerating scientific progress, yet current AI agents struggle with the complexities of rigorous, end-to-end experimentation. W…
ColBERT-serve: Efficient Multi-Stage Memory-Mapped Scoring
Kaili Huang, Thejas Venkatesh, Uma Dingankar +9
We study serving retrieval models, specifically late interaction models like ColBERT, to many concurrent users at once and under a small budget, in which the index may not fit in m…
LangProBe: a Language Programs Benchmark
Shangyin Tan, Lakshya A Agrawal, Arnav Singhvi +6
Composing language models (LMs) into multi-step language programs and automatically optimizing their modular prompts is now a mainstream paradigm for building AI systems, but the t…
WARP: An Efficient Engine for Multi-Vector Retrieval
Jan Luca Scheerer, Matei Zaharia, Christopher Potts +2
Multi-vector retrieval methods such as ColBERT and its recent variant, the ConteXtualized Token Retriever (XTR), offer high accuracy but face efficiency challenges at scale. To add…
ElasticTok: Adaptive Tokenization for Image and Video
Wilson Yan, Volodymyr Mnih, Aleksandra Faust +3
Efficient video tokenization remains a key bottleneck in learning general purpose vision models that are capable of processing long video sequences. Prevailing approaches are restr…
Optimizing Instructions and Demonstrations for Multi-Stage Language Model Programs
Krista Opsahl-Ong, Michael J Ryan, Josh Purtell +4
Language Model Programs, i.e. sophisticated pipelines of modular language model (LM) calls, are increasingly advancing NLP tasks, but they require crafting prompts that are jointly…
ScenicNL: Generating Probabilistic Scenario Programs from Natural Language
Karim Elmaaroufi, Devan Shanker, Ana Cismaru +4
For cyber-physical systems (CPS), including robotics and autonomous vehicles, mass deployment has been hindered by fatal errors that occur when operating in rare events. To replica…
World Model on Million-Length Video And Language With Blockwise RingAttention
Hao Liu, Wilson Yan, Matei Zaharia +1
Enabling long-context understanding remains a key challenge in scaling existing sequence models -- a crucial component in developing generally intelligent models that can process a…
: Learning Automata from Examples using Natural Language Oracles
Marcell Vazquez-Chanlatte, Karim Elmaaroufi, Stefan J. Witwicki +2
Expert demonstrations have proven an easy way to indirectly specify complex tasks. Recent algorithms even support extracting unambiguous formal specifications, e.g. deterministic f…
DSPy Assertions: Computational Constraints for Self-Refining Language Model Pipelines
Arnav Singhvi, Manish Shetty, Shangyin Tan +4
Chaining language model (LM) calls as composable modules is fueling a new way of programming, but ensuring LMs adhere to important constraints requires heuristic "prompt engineerin…
Ring Attention with Blockwise Transformers for Near-Infinite Context
Hao Liu, Matei Zaharia, Pieter Abbeel
Transformers have emerged as the architecture of choice for many state-of-the-art AI models, showcasing exceptional performance across a wide range of AI applications. However, the…
Exploration with Principles for Diverse AI Supervision
Hao Liu, Matei Zaharia, Pieter Abbeel
Training large transformers using next-token prediction has given rise to groundbreaking advancements in AI. While this generative AI approach has produced impressive results, it h…
ARES: An Automated Evaluation Framework for Retrieval-Augmented Generation Systems
Jon Saad-Falcon, Omar Khattab, Christopher Potts +1
Evaluating retrieval-augmented generation (RAG) systems traditionally relies on hand annotations for input queries, passages to retrieve, and responses to generate. We introduce AR…
DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines
Omar Khattab, Arnav Singhvi, Paridhi Maheshwari +10
The ML community is rapidly exploring techniques for prompting language models (LMs) and for stacking them into pipelines that solve complex tasks. Unfortunately, existing LM pipel…
Leveraging Large (Visual) Language Models for Robot 3D Scene Understanding
William Chen, Siyi Hu, Rajat Talak +1
Abstract semantic 3D scene understanding is a problem of critical importance in robotics. As robots still lack the common-sense knowledge about household objects and locations of a…