6 citations · 9 across the 5 of their papers we have counts for
5 papers
Don't Claim Benchmark-Oriented Optimization Improves General Coding Capability -- Diverse Evaluation Is Required
Egor Shibaev, Vera Kudrevskaia, Timur Galimzyanov +9
Post-training papers, model cards, and blog posts often treat scores on a small set of coding benchmarks (e.g., SWE-bench and LiveCodeBench) as evidence of broad coding capability,…
Practical Code RAG at Scale: Task-Aware Retrieval Design Choices under Compute Budgets
Timur Galimzyanov, Olga Kolomyttseva, Egor Bogomolov
We study retrieval design for code-focused generation tasks under realistic compute budgets. Using two complementary tasks from Long Code Arena -- code completion and bug localizat…
Drawing Pandas: A Benchmark for LLMs in Generating Plotting Code
Timur Galimzyanov, Sergey Titov, Yaroslav Golubev +1
This paper introduces the human-curated PandasPlotBench dataset, designed to evaluate language models' effectiveness as assistants in visual data exploration. Our benchmark focuses…
Long Code Arena: a Set of Benchmarks for Long-Context Code Models
Egor Bogomolov, Aleksandra Eliseeva, Timur Galimzyanov +8
Nowadays, the fields of code and natural language processing are evolving rapidly. In particular, models become better at processing long context windows - supported context sizes…
Dynamic Retrieval-Augmented Generation
Anton Shapkin, Denis Litvinov, Yaroslav Zharov +3
Current state-of-the-art large language models are effective in generating high-quality text and encapsulating a broad spectrum of world knowledge. These models, however, often hal…