4 papers
SWE-Effi: Re-Evaluating Software AI Agent System Effectiveness Under Resource Constraints
Zhiyu Fan, Kirill Vasilevski, Dayi Lin +6
The advancement of large language models (LLMs) and code agents has demonstrated significant potential to assist software engineering (SWE) tasks, such as autonomous issue resoluti…
The Hitchhikers Guide to Production-ready Trustworthy Foundation Model powered Software (FMware)
Kirill Vasilevski, Benjamin Rombaut, Gopi Krishnan Rajbahadur +10
Foundation Models (FMs) such as Large Language Models (LLMs) are reshaping the software industry by enabling FMware, systems that integrate these FMs as core components. In this KD…
Real-time Adapting Routing (RAR): Improving Efficiency Through Continuous Learning in Software Powered by Layered Foundation Models
Kirill Vasilevski, Dayi Lin, Ahmed E. Hassan
To balance the quality and inference cost of a Foundation Model (FM, such as large language models (LLMs)) powered software, people often opt to train a routing model that routes r…
Watson: A Cognitive Observability Framework for the Reasoning of LLM-Powered Agents
Benjamin Rombaut, Sogol Masoumzadeh, Kirill Vasilevski +2
Large language models (LLMs) are increasingly integrated into autonomous systems, giving rise to a new class of software known as Agentware, where LLM-powered agents perform comple…