4 papers
The Harness Effect: How Orchestration Design Sets the Token Economics of Enterprise Agentic AI
Muayad Sayed Ali, Aliaksandra Novik, Anji Boddupally +29
Agentic AI development today runs on token maxing: buying capability with tokens -- longer reasoning traces, more turns, wider tool payloads, bigger replayed contexts -- so tokens…
Towards Outcome-Oriented, Task-Agnostic Evaluation of AI Agents
Waseem AlShikh, Muayad Sayed Ali, Brian Kennedy +1
As AI agents proliferate across industries and applications, evaluating their performance based solely on infrastructural metrics such as latency, time-to-first-token, or token thr…
Reflect, Retry, Reward: Self-Improving LLMs via Reinforcement Learning
Shelly Bensal, Umar Jamil, Christopher Bryant +5
We explore a method for improving the performance of large language models through self-reflection and reinforcement learning. By incentivizing the model to generate better self-re…
Expect the Unexpected: FailSafe Long Context QA for Finance
Kiran Kamble, Melisa Russak, Dmytro Mozolevskyi +3
We propose a new long-context financial benchmark, FailSafeQA, designed to test the robustness and context-awareness of LLMs against six variations in human-interface interactions…