5 papers
Continual Learning Bench: Evaluating Frontier AI Systems in Real-World Stateful Environments
Parth Asawa, Christopher M. Glaze, Gabriel Orlanski +7
Continual learning, the ability of AI systems to improve through sequential experience, has attracted substantial interest, but no high-quality benchmark exists to evaluate it. We…
How to Train Your Advisor: Steering Black-Box LLMs with Advisor Models
Parth Asawa, Alan Zhu, Abigail O'Neill +3
Frontier language models are deployed as black-box services, where model weights cannot be modified and customization is limited to prompting. We introduce Advisor Models, a method…
SIEVE: Sample-Efficient Parametric Learning from Natural Language
Parth Asawa, Alexandros G. Dimakis, Matei Zaharia
Natural language context-such as instructions, knowledge, or feedback-contains rich signal for adapting language models. While in-context learning provides adaptation via the promp…
BARE: Leveraging Base Language Models for Few-Shot Synthetic Data Generation
Alan Zhu, Parth Asawa, Jared Quincy Davis +5
As the demand for high-quality data in model training grows, researchers and developers are increasingly generating synthetic data to tune and train LLMs. However, current data gen…
Semantic Operators: A Declarative Model for Rich, AI-based Data Processing
Liana Patel, Siddharth Jha, Melissa Pan +4
The semantic capabilities of large language models (LLMs) have the potential to enable rich analytics and reasoning over vast knowledge corpora. Unfortunately, existing systems eit…