4 papers
RubricRefine: Improving Tool-Use Agent Reliability with Training-Free Pre-Execution Refinement
Will LeVine, Brendan Evers, Sam Saltwick +1
Iterative self-refinement is a popular inference-time reliability technique, but its effectiveness in code-mode tool use depends heavily on the structure of the feedback signal: un…
Relevance Isn't All You Need: Scaling RAG Systems With Inference-Time Compute Via Multi-Criteria Reranking
Will LeVine, Bijan Varjavand
Modern Large Language Model (LLM) systems typically rely on Retrieval Augmented Generation (RAG) which aims to gather context that is useful for response generation. These RAG syst…
Out-of-Distribution Detection & Applications With Ablated Learned Temperature Energy
Will LeVine, Benjamin Pikus, Jacob Phillips +3
As deep neural networks become adopted in high-stakes domains, it is crucial to identify when inference inputs are Out-of-Distribution (OOD) so that users can be alerted of likely…
A Baseline Analysis of Reward Models' Ability To Accurately Analyze Foundation Models Under Distribution Shift
Will LeVine, Benjamin Pikus, Anthony Chen +1
Foundation models, specifically Large Language Models (LLMs), have lately gained wide-spread attention and adoption. Reinforcement Learning with Human Feedback (RLHF) involves trai…