10 papers
Performance, Efficiency and Collapse -- Advantages and Challenges in Offline Post-training of Code LLMs
Abhinav Anand, Sanjana Reddy Pachika, Shweta Verma +1
Post-training with reinforcement learning (RL) is a critical phase in the development of code-generating large language models (LLMs), as it ensures adherence to instructions and t…
SHERLOC: Structured Diagnostic Localization for Code Repair Agents
Hovhannes Tamoyan, Sean Narenthiran, Erik Arakelyan +2
LLM agents solve repository-level coding tasks through multi-turn tool use, but utilize half their budget on locating faults before editing. Dedicated localization frameworks have…
Efficient Post-training of LLMs for Code Generation With Offline Reinforcement Learning
Mingze Wu, Abhinav Anand, Shweta Verma +1
Post-training using online reinforcement learning (RL) is an important training step for LLMs, including code-generating models. However, online RL for code generation involves LLM…
Domain-Adaptable Reinforcement Learning for Code Generation with Dense Rewards
Erfan Aghadavoodi Jolfaei, Daniel Maninger, Abhinav Anand +2
Large language models show strong potential for automated code generation, but lack guarantees for correctness, quality, safety, and domain-specific constraints. For instance in ro…
Beyond BLEU: A Semantic Evaluation Method for Code Translation
Julius Näumann, Sven Keidel, Amir Molzam Sharifloo +1
Code translation is one of the core capabilities of LLMs. However, evaluating the correctness of translations remains difficult, as commonly used metrics such as BLEU measure only…
Deep Graph-Language Fusion for Structure-Aware Code Generation
Mert Tiftikci, Amir Molzam Sharifloo, Mira Mezini
Pre-trained Language Models (PLMs) have the potential to transform software development tasks. However, despite significant advances, current PLMs struggle to capture the structure…
Towards Understanding What State Space Models Learn About Code
Jiali Wu, Abhinav Anand, Shweta Verma +1
State Space Models (SSMs) have emerged as an efficient alternative to the Transformer architecture. Prior work shows that, when trained under comparable conditions, SSMs can match…
Analysis of Long Range Dependency Understanding in State Space Models
Srividya Ravikumar, Abhinav Anand, Shweta Verma +1
Although state-space models (SSMs) have demonstrated strong performance on long-sequence benchmarks, most research has emphasized predictive accuracy rather than interpretability.…
Integrating Symbolic Execution into the Fine-Tuning of Code-Generating LLMs
Marina Sakharova, Abhinav Anand, Mira Mezini
Code-generating Large Language Models (LLMs) have become essential tools in modern software development, enhancing productivity and accelerating development. This paper aims to inv…
CodeSSM: Towards State Space Models for Code Understanding
Shweta Verma, Abhinav Anand, Mira Mezini
Although transformers dominate many code-specific tasks, they have significant limitations. This paper explores State Space Models (SSMs) as a promising alternative for code unders…