17 papers
Modular Cognitive Architecture Emerges in Large Language Models
Pengrui Han, Jacob Andreas, Evelina Fedorenko +1
The human brain exhibits a striking degree of functional specialization, with distinct networks supporting language, formal reasoning, reasoning about other minds, and reasoning ab…
Towards Evaluation of Implicit Software World Models in Coding LLMs
Egor Bogomolov, Yaroslav Zharov
Software engineering, whether performed by humans or by AI agents, requires reasoning about how software behaves. We call the internal model that supports such reasoning the softwa…
LibEvoBench: Probing Temporal Knowledge Stratification in Code Generation Models
Daniele Cipollone, Sergey Titov, Maliheh Izadi +2
Large software projects often depend on older versions of libraries, even as APIs continue to evolve across releases. This creates a challenge for LLMs: they must maintain knowledg…
On Problems of Implicit Context Compression for Software Engineering Agents
Kirill Gelvan, Igor Slinko, Felix Steinbauer +3
LLM-based Software Engineering agents face a critical bottleneck: context length limitations cause failures on complex, long-horizon tasks. One promising solution is to encode cont…
Step Rejection Fine-Tuning: A Practical Distillation Recipe
Igor Slinko, Ilia Zavidnyi, Egor Bogomolov +1
Rejection Fine-Tuning (RFT) is a standard method for training LLM agents, where unsuccessful trajectories are discarded from the training set. In the context of SWE-bench tasks, th…
From Where Words Come: Efficient Regularization of Code Tokenizers Through Source Attribution
Pavel Chizhov, Egor Bogomolov, Ivan P. Yamshchikov
Efficiency and safety of Large Language Models (LLMs), among other factors, rely on the quality of tokenization. A good tokenizer not only improves inference speed and language und…