3 papers
cs.LG2026
Don't Claim Benchmark-Oriented Optimization Improves General Coding Capability -- Diverse Evaluation Is Required
Egor Shibaev, Vera Kudrevskaia, Timur Galimzyanov +9
Post-training papers, model cards, and blog posts often treat scores on a small set of coding benchmarks (e.g., SWE-bench and LiveCodeBench) as evidence of broad coding capability,…
cs.LG2025
Practical Code RAG at Scale: Task-Aware Retrieval Design Choices under Compute Budgets
Timur Galimzyanov, Olga Kolomyttseva, Egor Bogomolov
We study retrieval design for code-focused generation tasks under realistic compute budgets. Using two complementary tasks from Long Code Arena -- code completion and bug localizat…
cs.SE2024
Drawing Pandas: A Benchmark for LLMs in Generating Plotting Code
Timur Galimzyanov, Sergey Titov, Yaroslav Golubev +1
This paper introduces the human-curated PandasPlotBench dataset, designed to evaluate language models' effectiveness as assistants in visual data exploration. Our benchmark focuses…