3 papers
cs.AI2026
AgentLens: Production-Assessed Trajectory Reviews for Coding Agent Evaluation
Andrey Podivilov, Vadim Lomshakov, Sergey Savin +4
We present AgentLens, a production-assessed benchmark for interactive code agents. Most code-agent benchmarks reduce a run to a single bit -- did the task pass? -- but the people w…
cs.CL2025
YABLoCo: Yet Another Benchmark for Long Context Code Generation
Aidar Valeev, Roman Garaev, Vadim Lomshakov +3
Large Language Models demonstrate the ability to solve various programming tasks, including code generation. Typically, the performance of LLMs is measured on benchmarks with small…
cs.SE2025
LAMeD: LLM-generated Annotations for Memory Leak Detection
Ekaterina Shemetova, Ilya Shenbin, Ivan Smirnov +5
Static analysis tools are widely used to detect software bugs and vulnerabilities but often struggle with scalability and efficiency in complex codebases. Traditional approaches re…