33 citations · 52 across the 7 of their papers we have counts for
7 papers
HyDRA: Hybrid Dynamic Routing Architecture for Heterogeneous LLM Pools
Aashna Garg, Siddharth Singha Roy, Jinu Jang +2
Production LLM deployments increasingly maintain heterogeneous model pools spanning order-of-magnitude cost differences. Existing routers make binary strong-vs-weak decisions and c…
SetupBench: Assessing Software Engineering Agents' Ability to Bootstrap Development Environments
Avi Arora, Jinu Jang, Roshanak Zilouchian Moghaddam
Modern Large Language Model (LLM) agents promise end to end assistance with real-world software tasks, yet existing benchmarks evaluate LLM agents almost exclusively in pre-baked e…
RefactorBench: Evaluating Stateful Reasoning in Language Agents Through Code
Dhruv Gautam, Spandan Garg, Jinu Jang +2
Recent advances in language model (LM) agents and function calling have enabled autonomous, feedback-driven systems to solve problems across various digital domains. To better unde…
AutoDev: Automated AI-Driven Development
Michele Tufano, Anisha Agarwal, Jinu Jang +2
The landscape of software development has witnessed a paradigm shift with the advent of AI-powered assistants, exemplified by GitHub Copilot. However, existing solutions are not le…
Copilot Evaluation Harness: Evaluating LLM-Guided Software Programming
Anisha Agarwal, Aaron Chan, Shubham Chandel +6
The integration of Large Language Models (LLMs) into Development Environments (IDEs) has become a focal point in modern software development. LLMs such as OpenAI GPT-3.5/4 and Code…
AdaptivePaste: Code Adaptation through Learning Semantics-aware Variable Usage Representations
Xiaoyu Liu, Jinu Jang, Neel Sundaresan +2
In software development, it is common for programmers to copy-paste or port code snippets and then adapt them to their use case. This scenario motivates the code adaptation task --…
Program Merge Conflict Resolution via Neural Transformers
Alexey Svyatkovskiy, Sarah Fakhoury, Negar Ghorbani +6
Collaborative software development is an integral part of the modern software development life cycle, essential to the success of large-scale software projects. When multiple devel…