115 citations · 195 across the 32 of their papers we have counts for
5 papers · 2 filters
Diff-XYZ: A Benchmark for Evaluating Diff Understanding
Evgeniy Glukhov, Michele Conti, Egor Bogomolov +2
Reliable handling of code diffs is central to agents that edit and refactor repositories at scale. We introduce Diff-XYZ, a compact benchmark for code-diff understanding with three…
Challenge on Optimization of Context Collection for Code Completion
Dmitry Ustalov, Egor Bogomolov, Alexander Bezzubov +4
The rapid advancement of workflows and methods for software engineering using AI emphasizes the need for a systematic evaluation and analysis of their ability to leverage informati…
TreeRanker: Fast and Model-agnostic Ranking System for Code Suggestions in IDEs
Daniele Cipollone, Egor Bogomolov, Arie van Deursen +1
Token-level code completion is one of the most critical features in modern Integrated Development Environments (IDEs). It assists developers by suggesting relevant identifiers and…
The Complexity Trap: Simple Observation Masking Is as Efficient as LLM Summarization for Agent Context Management
Tobias Lindenbauer, Igor Slinko, Ludwig Felder +2
Large Language Model (LLM)-based agents solve complex tasks through iterative reasoning, exploration, and tool-use, a process that can result in long, expensive context histories.…
GitGoodBench: A Novel Benchmark For Evaluating Agentic Performance On Git
Tobias Lindenbauer, Egor Bogomolov, Yaroslav Zharov
Benchmarks for Software Engineering (SE) AI agents, most notably SWE-bench, have catalyzed progress in programming capabilities of AI agents. However, they overlook critical develo…