430 citations · 2.2k across the 34 of their papers we have counts for
14 papers · 1 filter
Disproving Program Equivalence with LLMs
Miltiadis Allamanis, Pengcheng Yin
To evaluate large language models (LLMs) for code, research has used manually created unit test-based benchmarks. However, these tests are often inadequate, missing corner cases an…
Unsupervised Evaluation of Code LLMs with Round-Trip Correctness
Miltiadis Allamanis, Sheena Panthaplackel, Pengcheng Yin
To evaluate code large language models (LLMs), research has relied on a few small manually curated benchmarks, such as HumanEval and MBPP, which represent a narrow part of the real…
JEMMA: An Extensible Java Dataset for ML4Code Applications
Anjan Karmakar, Miltiadis Allamanis, Romain Robbes
Machine Learning for Source Code (ML4Code) is an active research field in which extensive experimentation is needed to discover how to best use source code's richly structured info…
AdaptivePaste: Code Adaptation through Learning Semantics-aware Variable Usage Representations
Xiaoyu Liu, Jinu Jang, Neel Sundaresan +2
In software development, it is common for programmers to copy-paste or port code snippets and then adapt them to their use case. This scenario motivates the code adaptation task --…
Fast and Memory-Efficient Neural Code Completion
Alexey Svyatkovskiy, Sebastian Lee, Anna Hadjitofi +3
Code completion is one of the most widely used features of modern integrated development environments (IDEs). While deep learning has made significant progress in the statistical p…
Learning to Encode and Classify Test Executions
Foivos Tsimpourlas, Ajitha Rajan, Miltiadis Allamanis
The challenge of automatically determining the correctness of test executions is referred to as the test oracle problem and is one of the key remaining issues for automated testing…
DIRE: A Neural Approach to Decompiled Identifier Naming
Jeremy Lacomis, Pengcheng Yin, Edward J. Schwartz +4
The decompiler is one of the most common tools for examining binaries without corresponding source code. It transforms binaries into high-level code, reversing the compilation proc…
The Adverse Effects of Code Duplication in Machine Learning Models of Code
Miltiadis Allamanis
The field of big code relies on mining large corpora of code to perform some learning task. A significant threat to this approach has been recently identified by Lopes et al. (2017…
CODIT: Code Editing with Tree-Based Neural Models
Saikat Chakraborty, Yangruibo Ding, Miltiadis Allamanis +1
The way developers edit day-to-day code tends to be repetitive, often using existing code elements. Many researchers have tried to automate repetitive code changes by learning from…
A Survey of Machine Learning for Big Code and Naturalness
Miltiadis Allamanis, Earl T. Barr, Premkumar Devanbu +1
Research at the intersection of machine learning, programming languages, and software engineering has recently taken important steps in proposing learnable probabilistic models of…
Tailored Mutants Fit Bugs Better
Miltiadis Allamanis, Earl T. Barr, René Just +1
Mutation analysis measures test suite adequacy, the degree to which a test suite detects seeded faults: one test suite is better than another if it detects more mutants. Mutation a…
Mining Idioms from Source Code
Miltiadis Allamanis, Charles Sutton
We present the first method for automatically mining code idioms from a corpus of previously written, idiomatic software projects. We take the view that a code idiom is a syntactic…
Autofolding for Source Code Summarization
Jaroslav Fowkes, Pankajan Chanthirasegaran, Razvan Ranca +3
Developers spend much of their time reading and browsing source code, raising new opportunities for summarization methods. Indeed, modern code editors provide code folding, which a…
Learning Natural Coding Conventions
Miltiadis Allamanis, Earl T. Barr, Christian Bird +1
Every programmer has a characteristic style, ranging from preferences about identifier naming to preferences about object relationships and design patterns. Coding conventions defi…