430 citations · 2.2k across the 34 of their papers we have counts for
36 papers
Disproving Program Equivalence with LLMs
Miltiadis Allamanis, Pengcheng Yin
To evaluate large language models (LLMs) for code, research has used manually created unit test-based benchmarks. However, these tests are often inadequate, missing corner cases an…
NExT: Teaching Large Language Models to Reason about Code Execution
Ansong Ni, Miltiadis Allamanis, Arman Cohan +4
A fundamental skill among human developers is the ability to understand and reason about program execution. As an example, a programmer can mentally simulate code execution in natu…
Unsupervised Evaluation of Code LLMs with Round-Trip Correctness
Miltiadis Allamanis, Sheena Panthaplackel, Pengcheng Yin
To evaluate code large language models (LLMs), research has relied on a few small manually curated benchmarks, such as HumanEval and MBPP, which represent a narrow part of the real…
Epicure: Distilling Sequence Model Predictions into Patterns
Miltiadis Allamanis, Earl T. Barr
Most machine learning models predict a probability distribution over concrete outputs and struggle to accurately predict names over high entropy sequence distributions. Here, we ex…
JEMMA: An Extensible Java Dataset for ML4Code Applications
Anjan Karmakar, Miltiadis Allamanis, Romain Robbes
Machine Learning for Source Code (ML4Code) is an active research field in which extensive experimentation is needed to discover how to best use source code's richly structured info…
Overwatch: Learning Patterns in Code Edit Sequences
Yuhao Zhang, Yasharth Bajpai, Priyanshu Gupta +8
Integrated Development Environments (IDEs) provide tool support to automate many source code editing tasks. Traditionally, IDEs use only the spatial context, i.e., the location whe…
AdaptivePaste: Code Adaptation through Learning Semantics-aware Variable Usage Representations
Xiaoyu Liu, Jinu Jang, Neel Sundaresan +2
In software development, it is common for programmers to copy-paste or port code snippets and then adapt them to their use case. This scenario motivates the code adaptation task --…
NS3: Neuro-Symbolic Semantic Code Search
Shushan Arakelyan, Anna Hakhverdyan, Miltiadis Allamanis +3
Semantic code search is the task of retrieving a code snippet given a textual description of its functionality. Recent work has been focused on using similarity metrics between neu…
Is Surprisal in Issue Trackers Actionable?
James Caddy, Markus Wagner, Christoph Treude +2
Background. From information theory, surprisal is a measurement of how unexpected an event is. Statistical language models provide a probabilistic approximation of natural language…
HEAT: Hyperedge Attention Networks
Dobrik Georgiev, Marc Brockschmidt, Miltiadis Allamanis
Learning from structured data is a core machine learning task. Commonly, such data is represented as graphs, which normally only consider (typed) binary relationships between pairs…
CoRGi: Content-Rich Graph Neural Networks with Attention
Jooyeon Kim, Angus Lamb, Simon Woodhead +3
Graph representations of a target domain often project it to a set of entities (nodes) and their relations (edges). However, such projections often miss important and rich informat…
Learning to Complete Code with Sketches
Daya Guo, Alexey Svyatkovskiy, Jian Yin +3
Code completion is usually cast as a language modelling problem, i.e., continuing an input in a left-to-right fashion. However, in practice, some parts of the completion (e.g., str…
Self-Supervised Bug Detection and Repair
Miltiadis Allamanis, Henry Jackson-Flux, Marc Brockschmidt
Machine learning-based program analyses have recently shown the promise of integrating formal and probabilistic reasoning towards aiding software development. However, in the absen…
Copy that! Editing Sequences by Copying Spans
Sheena Panthaplackel, Miltiadis Allamanis, Marc Brockschmidt
Neural sequence-to-sequence models are finding increasing use in editing of documents, for example in correcting a text document or repairing source code. In this paper, we argue t…
Typilus: Neural Type Hints
Miltiadis Allamanis, Earl T. Barr, Soline Ducousso +1
Type inference over partial contexts in dynamically typed languages is challenging. In this work, we present a graph neural network model that predicts types by probabilistically r…
Fast and Memory-Efficient Neural Code Completion
Alexey Svyatkovskiy, Sebastian Lee, Anna Hadjitofi +3
Code completion is one of the most widely used features of modern integrated development environments (IDEs). While deep learning has made significant progress in the statistical p…
Learning to Encode and Classify Test Executions
Foivos Tsimpourlas, Ajitha Rajan, Miltiadis Allamanis
The challenge of automatically determining the correctness of test executions is referred to as the test oracle problem and is one of the key remaining issues for automated testing…
DIRE: A Neural Approach to Decompiled Identifier Naming
Jeremy Lacomis, Pengcheng Yin, Edward J. Schwartz +4
The decompiler is one of the most common tools for examining binaries without corresponding source code. It transforms binaries into high-level code, reversing the compilation proc…
CodeSearchNet Challenge: Evaluating the State of Semantic Code Search
Hamel Husain, Ho-Hsiang Wu, Tiferet Gazit +2
Semantic code search is the task of retrieving relevant code given a natural language query. While related to other information retrieval tasks, it requires bridging the gap betwee…
Program Synthesis and Semantic Parsing with Learned Code Idioms
Richard Shin, Miltiadis Allamanis, Marc Brockschmidt +1
Program synthesis of general-purpose source code from natural language specifications is challenging due to the need to reason about high-level patterns in the target program and l…
The Adverse Effects of Code Duplication in Machine Learning Models of Code
Miltiadis Allamanis
The field of big code relies on mining large corpora of code to perform some learning task. A significant threat to this approach has been recently identified by Lopes et al. (2017…
Structured Neural Summarization
Patrick Fernandes, Miltiadis Allamanis, Marc Brockschmidt
Summarization of long sequences into a concise statement is a core problem in natural language processing, requiring non-trivial understanding of the input. Based on the promising…
Learning to Represent Edits
Pengcheng Yin, Graham Neubig, Miltiadis Allamanis +2
We introduce the problem of learning distributed representations of edits. By combining a "neural editor" with an "edit encoder", our models learn to represent the salient informat…
CODIT: Code Editing with Tree-Based Neural Models
Saikat Chakraborty, Yangruibo Ding, Miltiadis Allamanis +1
The way developers edit day-to-day code tends to be repetitive, often using existing code elements. Many researchers have tried to automate repetitive code changes by learning from…