183 citations · 432 across the 25 of their papers we have counts for
28 papers
Joint Dropout: Improving Generalizability in Low-Resource Neural Machine Translation through Phrase Pair Variables
Ali Araabi, Vlad Niculae, Christof Monz
Despite the tremendous success of Neural Machine Translation (NMT), its performance on low-resource language pairs still remains subpar, partly due to the limited ability to handle…
Two derivations of Principal Component Analysis on datasets of distributions
Vlad Niculae
In this brief note, we formulate Principal Component Analysis (PCA) over datasets consisting not of points but of distributions, characterized by their location and covariance. Jus…
Viewing Knowledge Transfer in Multilingual Machine Translation Through a Representational Lens
David Stap, Vlad Niculae, Christof Monz
We argue that translation quality alone is not a sufficient metric for measuring knowledge transfer in multilingual neural machine translation. To support this claim, we introduce…
DAG Learning on the Permutahedron
Valentina Zantedeschi, Luca Franceschi, Jean Kaddour +2
We propose a continuous optimization framework for discovering a latent directed acyclic graph (DAG) from observational data. Our approach optimizes over the polytope of permutatio…
Discrete Latent Structure in Neural Networks
Vlad Niculae, Caio F. Corro, Nikita Nangia +2
Many types of data from fields including natural language processing, computer vision, and bioinformatics, are well represented by discrete, compositional structures such as trees,…
How Effective is Byte Pair Encoding for Out-Of-Vocabulary Words in Neural Machine Translation?
Ali Araabi, Christof Monz, Vlad Niculae
Neural Machine Translation (NMT) is an open vocabulary problem. As a result, dealing with the words not occurring during training (a.k.a. out-of-vocabulary (OOV) words) have long b…
Modeling Structure with Undirected Neural Networks
Tsvetomila Mihaylova, Vlad Niculae, André F. T. Martins
Neural networks are powerful function estimators, leading to their status as a paradigm of choice for modeling structured data. However, unlike other structured representations tha…
Sparse Communication via Mixed Distributions
António Farinhas, Wilker Aziz, Vlad Niculae +1
Neural networks and other machine learning models compute continuous representations, while humans communicate mostly through discrete symbols. Reconciling these two forms of commu…
Sparse Continuous Distributions and Fenchel-Young Losses
André F. T. Martins, Marcos Treviso, António Farinhas +4
Exponential families are widely used in machine learning, including many distributions in continuous and discrete domains (e.g., Gaussian, Dirichlet, Poisson, and categorical distr…
Understanding the Mechanics of SPIGOT: Surrogate Gradients for Latent Structure Learning
Tsvetomila Mihaylova, Vlad Niculae, André F. T. Martins
Latent structure models are a powerful tool for modeling language data: they can mitigate the error propagation and annotation bottleneck in pipeline systems, while simultaneously…
Learning Binary Decision Trees by Argmin Differentiation
Valentina Zantedeschi, Matt J. Kusner, Vlad Niculae
We address the problem of learning binary decision trees that partition data for some downstream task. We propose to learn discrete parameters (i.e., for tree traversals and node p…
Efficient Marginalization of Discrete and Structured Latent Variables via Sparsity
Gonçalo M. Correia, Vlad Niculae, Wilker Aziz +1
Training neural network models with discrete (categorical or structured) latent variables can be computationally challenging, due to the need for marginalization over large or comb…
Sparse and Continuous Attention Mechanisms
André F. T. Martins, António Farinhas, Marcos Treviso +3
Exponential families are widely used in machine learning; they include many distributions in continuous and discrete domains (e.g., Gaussian, Dirichlet, Poisson, and categorical di…
Sparse and Structured Visual Attention
Pedro Henrique Martins, Vlad Niculae, Zita Marinho +1
Visual attention mechanisms are widely used in multimodal tasks, as visual question answering (VQA). One drawback of softmax-based attention mechanisms is that they assign some pro…
LP-SparseMAP: Differentiable Relaxed Optimization for Sparse Structured Prediction
Vlad Niculae, André F. T. Martins
Structured prediction requires manipulating a large number of combinatorial structures, e.g., dependency trees or alignments, either as latent or output variables. Recently, the Sp…
Adaptively Sparse Transformers
Gonçalo M. Correia, Vlad Niculae, André F. T. Martins
Attention mechanisms have become ubiquitous in NLP. Recent architectures, notably the Transformer, learn powerful context-aware word representations through layered, multi-headed a…
Sparse Sequence-to-Sequence Models
Ben Peters, Vlad Niculae, André F. T. Martins
Sequence-to-sequence models are a powerful workhorse of NLP. Most variants employ a softmax transformation in both their attention mechanism and output layer, leading to dense alig…
Learning with Fenchel-Young Losses
Mathieu Blondel, André F. T. Martins, Vlad Niculae
Over the past decades, numerous loss functions have been been proposed for a variety of supervised learning tasks, including regression, classification, ranking, and more generally…
Towards Dynamic Computation Graphs via Sparse Latent Structure
Vlad Niculae, André F. T. Martins, Claire Cardie
Deep NLP models benefit from underlying structures in the data---e.g., parse trees---typically extracted using off-the-shelf parsers. Recent attempts to jointly learn the latent st…
Learning Classifiers with Fenchel-Young Losses: Generalized Entropies, Margins, and Algorithms
Mathieu Blondel, André F. T. Martins, Vlad Niculae
This paper studies Fenchel-Young losses, a generic way to construct convex loss functions from a regularization function. We analyze their properties in depth, showing that they un…
SparseMAP: Differentiable Sparse Structured Inference
Vlad Niculae, André F. T. Martins, Mathieu Blondel +1
Structured prediction requires searching over a combinatorial number of structures. To tackle it, we introduce SparseMAP: a new method for sparse structured inference, and its natu…
A Regularized Framework for Sparse and Structured Neural Attention
Vlad Niculae, Mathieu Blondel
Modern neural networks are often augmented with an attention mechanism, which tells the network where to focus within the input. We propose in this paper a new framework for sparse…
Multi-output Polynomial Networks and Factorization Machines
Mathieu Blondel, Vlad Niculae, Takuma Otsuka +1
Factorization machines and polynomial networks are supervised polynomial models based on an efficient low-rank decomposition. We extend these models to the multi-output setting, i.…
Argument Mining with Structured SVMs and RNNs
Vlad Niculae, Joonsuk Park, Claire Cardie
We propose a novel factor graph model for argument mining, designed for settings in which the argumentative relations in a document do not necessarily form a tree structure. (This…