5 papers
Better Harnesses, Smaller Models: Building 90% Cheaper Agents via Automated Harness Adaptation
Chenyang Yang, Xinran Zhao, Tongshuang Wu +1
Frontier LLM agents are automating many business tasks, but their high inference cost makes large-scale deployment unsustainable. Small language models (SLMs) offer a cheaper alter…
Towards Verifiably Safe Tool Use for LLM Agents
Aarya Doshi, Yining Hong, Congying Xu +3
Large language model (LLM)-based AI agents extend LLM capabilities by enabling access to tools such as data sources, APIs, search engines, code sandboxes, and even other agents. Wh…
From Hazard Identification to Controller Design: Proactive and LLM-Supported Safety Engineering for ML-Powered Systems
Yining Hong, Christopher S. Timperley, Christian Kästner
Machine learning (ML) components are increasingly integrated into software products, yet their complexity and inherent uncertainty often lead to unintended and hazardous consequenc…
Orbit: A Framework for Designing and Evaluating Multi-objective Rankers
Chenyang Yang, Tesi Xiao, Michael Shavlovsky +2
Machine learning in production needs to balance multiple objectives: This is particularly evident in ranking or recommendation models, where conflicting objectives such as user eng…
What Is Wrong with My Model? Identifying Systematic Problems with Semantic Data Slicing
Chenyang Yang, Yining Hong, Grace A. Lewis +2
Machine learning models make mistakes, yet sometimes it is difficult to identify the systematic problems behind the mistakes. Practitioners engage in various activities, including…