Publications (18)
KERMIT: Generative Insertion-Based Modeling for Sequences
William Chan, Nikita Kitaev, Kelvin Guu +2
We present KERMIT, a simple insertion-based approach to generative modeling for sequences and sequence pairs. KERMIT models the joint distribution and its decompositions (i.e., mar…
A Minimal Span-Based Neural Constituency Parser
Mitchell Stern, Jacob Andreas, Dan Klein
In this work, we present a minimal neural model for constituency parsing based on independent scoring of labels and spans. We show that this model is not only compatible with class…
Towards End-to-End In-Image Neural Machine Translation
Elman Mansimov, Mitchell Stern, Mia Chen +3
In this paper, we offer a preliminary investigation into the task of in-image machine translation: transforming an image containing text in one language into an image containing th…
Effective Inference for Generative Neural Parsing
Mitchell Stern, Daniel Fried, Dan Klein
Generative neural models have recently achieved state-of-the-art results for constituency parsing. However, without a feasible search procedure, their use has so far been limited t…
An Empirical Study of Generation Order for Machine Translation
William Chan, Mitchell Stern, Jamie Kiros +1
In this work, we present an empirical study of generation order for machine translation. Building on recent advances in insertion-based modeling, we first introduce a soft order-re…
Adafactor: Adaptive Learning Rates with Sublinear Memory Cost
Noam Shazeer, Mitchell Stern
In several recently proposed stochastic optimization methods (e.g. RMSProp, Adam, Adadelta), parameter updates are scaled by the inverse square roots of exponential moving averages…
Semantic Scaffolds for Pseudocode-to-Code Generation
Ruiqi Zhong, Mitchell Stern, Dan Klein
We propose a method for program generation based on semantic scaffolds, lightweight structures representing the high-level semantic and syntactic composition of a program. By first…
Insertion Transformer: Flexible Sequence Generation via Insertion Operations
Mitchell Stern, William Chan, Jamie Kiros +1
We present the Insertion Transformer, an iterative, partially autoregressive model for sequence generation based on insertion operations. Unlike typical autoregressive models which…
Kernel Feature Selection via Conditional Covariance Minimization
Jianbo Chen, Mitchell Stern, Martin J. Wainwright +1
We propose a method for feature selection that employs kernel-based measures of independence to find a subset of covariates that is maximally predictive of the response. Building o…
The Marginal Value of Adaptive Gradient Methods in Machine Learning
Ashia C. Wilson, Rebecca Roelofs, Mitchell Stern +2
Adaptive optimization methods, which perform local optimization with a metric constructed from the history of iterates, are becoming increasingly popular for training deep neural n…
Improving Neural Parsing by Disentangling Model Combination and Reranking Effects
Daniel Fried, Mitchell Stern, Dan Klein
Recent work has proposed several generative neural models for constituency parsing that achieve state-of-the-art results. Since direct search in these generative models is difficul…
What's Going On in Neural Constituency Parsers? An Analysis
David Gaddy, Mitchell Stern, Dan Klein
A number of differences have emerged between modern and classic approaches to constituency parsing in recent years, with structural components like grammars and feature-rich lexico…
Insertion-Deletion Transformer
Laura Ruis, Mitchell Stern, Julia Proskurnia +1
We propose the Insertion-Deletion Transformer, a novel transformer-based neural architecture and training method for sequence generation. The model consists of two phases that are…
Dynamic Posted-Price Mechanisms for the Blockchain Transaction Fee Market
Matheus V. X. Ferreira, Daniel J. Moroz, David C. Parkes +1
In recent years, prominent blockchain systems such as Bitcoin and Ethereum have experienced explosive growth in transaction volume, leading to frequent surges in demand for limited…
Blockwise Parallel Decoding for Deep Autoregressive Models
Mitchell Stern, Noam Shazeer, Jakob Uszkoreit
Deep autoregressive sequence-to-sequence models have demonstrated impressive performance across a wide variety of tasks in recent years. While common architecture classes such as r…
Stochastic Cubic Regularization for Fast Nonconvex Optimization
Nilesh Tripuraneni, Mitchell Stern, Chi Jin +2
This paper proposes a stochastic variant of a classic algorithm---the cubic-regularized Newton method [Nesterov and Polyak 2006]. The proposed algorithm efficiently escapes saddle…
Imitation Attacks and Defenses for Black-box Machine Translation Systems
Eric Wallace, Mitchell Stern, Dawn Song
Adversaries may look to steal or attack black-box NLP systems, either for financial gain or to exploit model errors. One setting of particular interest is machine translation (MT),…
Abstract Syntax Networks for Code Generation and Semantic Parsing
Maxim Rabinovich, Mitchell Stern, Dan Klein
Tasks like code generation and semantic parsing require mapping unstructured (or partially structured) inputs to well-formed, executable outputs. We introduce abstract syntax netwo…