7 papers
Beyond Isolated Tasks: A Framework for Evaluating Coding Agents on Sequential Software Evolution
KN Ajay Shastry, Ganesh Senrayan, Shrey Satapara +2
Existing datasets for coding agents evaluate performance on isolated, single pull request (PR) tasks in a stateless manner, failing to capture the reality of real-world software de…
EnterpriseLab: A Full-Stack Platform for developing and deploying agents in Enterprises
Ankush Agarwal, Harsh Vishwakarma, Suraj Nagaje +1
Deploying AI agents in enterprise environments requires balancing capability with data sovereignty and cost constraints. While small language models offer privacy-preserving altern…
Can LLMs Help You at Work? A Sandbox for Evaluating LLM Agents in Enterprise Environments
Harsh Vishwakarma, Ankush Agarwal, Ojas Patil +2
Enterprise systems are crucial for enhancing productivity and decision-making among employees and customers. Integrating LLM based systems into enterprise systems enables intellige…
Adaptive LLM Routing under Budget Constraints
Pranoy Panda, Raghav Magazine, Chaitanya Devaguptapu +2
Large Language Models (LLMs) have revolutionized natural language processing, but their varying capabilities and costs pose challenges in practical applications. LLM routing addres…
Finding Needles in Images: Can Multimodal LLMs Locate Fine Details?
Parth Thakkar, Ankush Agarwal, Prasad Kasu +2
While Multi-modal Large Language Models (MLLMs) have shown impressive capabilities in document understanding tasks, their ability to locate and reason about fine-grained details wi…
Hybrid Graphs for Table-and-Text based Question Answering using LLMs
Ankush Agarwal, Ganesh S, Chaitanya Devaguptapu
Answering questions that require reasoning and aggregation across both structured (tables) and unstructured (raw text) data sources presents significant challenges. Current methods…