14 citations · 31 across the 30 of their papers we have counts for
32 papers
MILO: Automated Harness Discovery via Orchestrated Multi-Agent Evolution
Prithwish Jana, Mononito Goswami, Hao Liu +9
Modern agentic systems combine an AI model with a harness that controls execution and environmental interactions. Harness design strongly affects long-horizon performance, yet its…
SWE-Proof: Can Language Models Resolve Real-World Issues with Machine-Checked Proofs?
George Ma, Benjamin Mikek, Haoyu Li +9
Ensuring the correctness of LLM-generated code is a core challenge for modern software engineering. Benchmarks for agentic code generation check correctness with held-out test suit…
Dissecting model behavior through agent trajectories
Gaurav Gupta, Vatshank Chaturvedi, Jun Huan +2
AI agent performance is not just a modeling problem, it is fundamentally a systems problem. The advanced capabilities of models are realized through agent harnesses. Therefore, a g…
ContextWeaver: Selective and Dependency-Structured Memory Construction for LLM Agents
Yating Wu, Yuhao Zhang, Sayan Ghosh +4
Large language model (LLM) agents often struggle in long-context interactions. As the history grows, sliding windows and prompt compression may drop earlier information that later…
ExecTune: Effective Steering of Black-Box LLMs with Guide Models
Vijay Lingam, Aditya Golatkar, Anwesan Pal +6
For large language models deployed through black-box APIs, recurring inference costs often exceed one-time training costs. This motivates composed agentic systems that amortize exp…
TerraFormer: Automated Infrastructure-as-Code with LLMs Fine-Tuned via Policy-Guided Verifier Feedback
Prithwish Jana, Sam Davidson, Bhavana Bhasker +3
Automating Infrastructure-as-Code (IaC) is challenging, and large language models (LLMs) often produce incorrect configurations from natural language (NL). We present TerraFormer,…
Empowering Multi-Turn Tool-Integrated Agentic Reasoning with Group Turn Policy Optimization
Yifeng Ding, Hung Le, Songyang Han +5
Training Large Language Models (LLMs) for multi-turn Tool-Integrated Reasoning (TIR) - where models iteratively reason, generate code, and verify through execution - remains challe…
MURPHY: Feedback-Aware GRPO with Retrospective Credit Assignment for Multi-Turn Code Generation
Chanakya Ekbote, Vijay Lingam, Sujay Sanghavi +4
Reinforcement Learning with Verifiable Rewards (RLVR) has become a standard recipe for post-training LLMs on reasoning tasks, with Group Relative Policy Optimization (GRPO) emergin…
Multi-IaC-Eval: Benchmarking Cloud Infrastructure as Code Across Multiple Formats
Sam Davidson, Li Sun, Bhavana Bhasker +2
Infrastructure as Code (IaC) is fundamental to modern cloud computing, enabling teams to define and manage infrastructure through machine-readable configuration files. However, dif…
Structural Code Search using Natural Language Queries
Ben Limpanukorn, Yanjun Wang, Zach Patterson +5
Searching code is a common task that developers perform to understand APIs, learn common code patterns, and navigate code. Currently, developers most commonly search using keywords…
CodeAssistBench (CAB): Dataset & Benchmarking for Multi-turn Chat-Based Code Assistance
Myeongsoo Kim, Shweta Garg, Baishakhi Ray +2
Programming assistants powered by large language models have improved dramatically, yet existing benchmarks still evaluate them in narrow code-generation settings. Recent efforts s…
The Amazon Nova Family of Models: Technical Report and Model Card
Amazon AGI, Aaron Langford, Aayush Shah +783
We present Amazon Nova, a new generation of state-of-the-art foundation models that deliver frontier intelligence and industry-leading price performance. Amazon Nova Pro is a highl…
Lossless Token Sequence Compression via Meta-Tokens
John Harvill, Ziwei Fan, Hao Wang +4
Existing work on prompt compression for Large Language Models (LLM) focuses on lossy methods that try to maximize the retention of semantic information that is relevant to downstre…
MigrationBench: Repository-Level Code Migration Benchmark from Java 8
Linbo Liu, Xinle Liu, Qiang Zhou +8
With the rapid advancement of powerful large language models (LLMs) in recent years, a wide range of software engineering tasks can now be addressed using LLMs, significantly enhan…
SWE-PolyBench: A multi-language benchmark for repository level evaluation of coding agents
Muhammad Shihab Rashid, Christian Bock, Yuan Zhuang +10
Coding agents powered by large language models have shown impressive capabilities in software engineering tasks, but evaluating their performance across diverse programming languag…
UTFix: Change Aware Unit Test Repairing using LLM
Shanto Rahman, Sachit Kuhar, Berk Cirisci +5
Software updates, including bug repair and feature additions, are frequent in modern applications but they often leave test suites outdated, resulting in undetected bugs and increa…
LibEvolutionEval: A Benchmark and Study for Version-Specific Code Generation
Sachit Kuhar, Wasi Uddin Ahmad, Zijian Wang +6
Recent advancements in code completion models have primarily focused on local file contexts. However, these studies do not fully capture the complexity of real-world software devel…
Approximately Aligned Decoding
Daniel Melcer, Sujan Gonugondla, Pramuditha Perera +7
It is common to reject undesired outputs of Large Language Models (LLMs); however, current methods to do so require an excessive amount of computation to re-sample after a rejectio…
Automated Evaluation of Retrieval-Augmented Language Models with Task-Specific Exam Generation
Gauthier Guinet, Behrooz Omidvar-Tehrani, Anoop Deoras +1
We propose a new method to measure the task-specific accuracy of Retrieval-Augmented Large Language Models (RAG). Evaluation is performed by scoring the RAG on an automatically-gen…
Collage: Light-Weight Low-Precision Strategy for LLM Training
Tao Yu, Gaurav Gupta, Karthick Gopalswamy +7
Large models training is plagued by the intense compute cost and limited hardware memory. A practical solution is low-precision representation but is troubled by loss in numerical…
Fewer Truncations Improve Language Modeling
Hantian Ding, Zijian Wang, Giovanni Paolini +4
In large language model training, input documents are typically concatenated together and then split into sequences of equal length to avoid padding tokens. Despite its efficiency,…
LeDex: Training LLMs to Better Self-Debug and Explain Code
Nan Jiang, Xiaopeng Li, Shiqi Wang +6
In the domain of code generation, self-debugging is crucial. It allows LLMs to refine their generated code based on execution feedback. This is particularly important because gener…
CodeFort: Robust Training for Code Generation Models
Yuhao Zhang, Shiqi Wang, Haifeng Qian +8
Code generation models are not robust to small perturbations, which often lead to incorrect generations and significantly degrade the performance of these models. Although improvin…
BASS: Batched Attention-optimized Speculative Sampling
Haifeng Qian, Sujan Kumar Gonugondla, Sungsoo Ha +6
Speculative decoding has emerged as a powerful method to improve latency and throughput in hosting large language models. However, most existing implementations focus on generating…