354 citations · 898 across the 23 of their papers we have counts for
25 papers
LLaMA-Reviewer: Advancing Code Review Automation with Large Language Models through Parameter-Efficient Fine-Tuning
Junyi Lu, Lei Yu, Xiaojia Li +2
The automation of code review activities, a long-standing pursuit in software engineering, has been primarily addressed by numerous domain-specific pre-trained models. Despite thei…
Word sense extension
Lei Yu, Yang Xu
Humans often make creative use of words to express novel senses. A long-standing effort in natural language processing has been focusing on word sense disambiguation (WSD), but lit…
A Reparameterized Discrete Diffusion Model for Text Generation
Lin Zheng, Jianbo Yuan, Lei Yu +1
This work studies discrete diffusion probabilistic models with applications to natural language generation. We derive an alternative yet equivalent formulation of the sampling from…
A Natural Bias for Language Generation Models
Clara Meister, Wojciech Stokowiec, Tiago Pimentel +3
After just a few hundred training updates, a standard probabilistic model for language generation has likely not yet learnt many semantic or syntactic rules of natural language, ma…
MAD for Robust Reinforcement Learning in Machine Translation
Domenic Donato, Lei Yu, Wang Ling +1
We introduce a new distributed policy gradient algorithm and show that it outperforms existing reward-aware training procedures such as REINFORCE, minimum risk training (MRT) and p…
Noun2Verb: Probabilistic frame semantics for word class conversion
Lei Yu, Yang Xu
Humans can flexibly extend word usages across different grammatical classes, a phenomenon known as word class conversion. Noun-to-verb conversion, or denominal verb (e.g., to Googl…
Enabling arbitrary translation objectives with Adaptive Tree Search
Wang Ling, Wojciech Stokowiec, Domenic Donato +4
We introduce an adaptive tree search algorithm, that can find high-scoring outputs under translation models that make no assumptions about the form or structure of the search objec…
Documentation based Semantic-Aware Log Parsing
Lei Yu, Tian Wu, Jiaqi Li +3
With the recent advances of deep learning techniques, there are rapidly growing interests in applying machine learning to log data. As a fundamental part of log analytics, accurate…
Serving Deep Learning Models with Deduplication from Relational Databases
Lixi Zhou, Jiaqing Chen, Amitabh Das +4
There are significant benefits to serve deep learning models from relational databases. First, features extracted from databases do not need to be transferred to any decoupled deep…
Predicting emergent linguistic compositions through time: Syntactic frame extension via multimodal chaining
Lei Yu, Yang Xu
Natural language relies on a finite lexicon to express an unbounded set of emerging ideas. One result of this tension is the formation of new compositions, such that existing lingu…
Diverse Pretrained Context Encodings Improve Document Translation
Domenic Donato, Lei Yu, Chris Dyer
We propose a new architecture for adapting a sentence-level sequence-to-sequence transformer by incorporating multiple pretrained document context signals and assess the impact on…
Pretraining the Noisy Channel Model for Task-Oriented Dialogue
Qi Liu, Lei Yu, Laura Rimell +1
Direct decoding for task-oriented dialogue is known to suffer from the explaining-away effect, manifested in models that prefer short and generic responses. Here we argue for the u…
Inferring symmetry in natural language
Chelsea Tanchip, Lei Yu, Aotao Xu +1
We present a methodological framework for inferring symmetry of verb predicates in natural language. Empirical work on predicate symmetry has taken two main approaches. The feature…
Capturing document context inside sentence-level neural machine translation models with self-training
Elman Mansimov, Gábor Melis, Lei Yu
Neural machine translation (NMT) has arguably achieved human level parity when trained and evaluated at the sentence-level. Document-level neural machine translation has received l…
A Mutual Information Maximization Perspective of Language Representation Learning
Lingpeng Kong, Cyprien de Masson d'Autume, Wang Ling +3
We show state-of-the-art word representation learning methods maximize an objective function that is a lower bound on the mutual information between different parts of a word seque…
Better Document-Level Machine Translation with Bayes' Rule
Lei Yu, Laurent Sartran, Wojciech Stokowiec +4
We show that Bayes' rule provides an effective mechanism for creating document translation models that can be learned from only parallel sentences and monolingual documents---a com…
Unsupervised Recurrent Neural Network Grammars
Yoon Kim, Alexander M. Rush, Lei Yu +3
Recurrent neural network grammars (RNNG) are generative models of language which jointly model syntax and surface structure by incrementally generating a syntax tree and sentence i…
Learning and Evaluating General Linguistic Intelligence
Dani Yogatama, Cyprien de Masson d'Autume, Jerome Connor +8
We define general linguistic intelligence as the ability to reuse previously acquired knowledge about a language's lexicon, syntax, semantics, and pragmatic conventions to adapt to…
Variational Smoothing in Recurrent Neural Network Language Models
Lingpeng Kong, Gabor Melis, Wang Ling +2
We present a new theoretical perspective of data noising in recurrent neural network language models (Xie et al., 2017). We show that each variant of data noising is an instance of…
Sentence Encoding with Tree-constrained Relation Networks
Lei Yu, Cyprien de Masson d'Autume, Chris Dyer +3
The meaning of a sentence is a function of the relations that hold between its words. We instantiate this relational view of semantics in a series of neural models based on variant…
Tackling Sequence to Sequence Mapping Problems with Neural Networks
Lei Yu
In Natural Language Processing (NLP), it is important to detect the relationship between two sequences or to generate a sequence of tokens given another observed sequence. We call…
The Neural Noisy Channel
Lei Yu, Phil Blunsom, Chris Dyer +2
We formulate sequence to sequence transduction as a noisy channel decoding problem and use recurrent neural networks to parameterise the source and channel models. Unlike direct mo…
Online Segment to Segment Neural Transduction
Lei Yu, Jan Buys, Phil Blunsom
We introduce an online neural sequence to sequence model that learns to alternate between encoding and decoding segments of the input as it is read. By independently tracking the e…
Neural Variational Inference for Text Processing
Yishu Miao, Lei Yu, Phil Blunsom
Recent advances in neural variational inference have spawned a renaissance in deep latent variable models. In this paper we introduce a generic variational inference framework for…