11 citations · 28 across the 10 of their papers we have counts for
12 papers
mPLM-Sim: Better Cross-Lingual Similarity and Transfer in Multilingual Pretrained Language Models
Peiqin Lin, Chengzhi Hu, Zheyu Zhang +2
Recent multilingual pretrained language models (mPLMs) have been shown to encode strong language-specific signals, which are not explicitly provided during pretraining. It remains…
Discrete Latent Structure in Neural Networks
Vlad Niculae, Caio F. Corro, Nikita Nangia +2
Many types of data from fields including natural language processing, computer vision, and bioinformatics, are well represented by discrete, compositional structures such as trees,…
Learning to Scaffold: Optimizing Model Explanations for Teaching
Patrick Fernandes, Marcos Treviso, Danish Pruthi +2
Modern machine learning models are opaque, and as a result there is a burgeoning academic subfield on methods that explain these models' behavior. However, what is the precise goal…
Differentiable Causal Discovery Under Latent Interventions
Gonçalo R. A. Faria, André F. T. Martins, Mário A. T. Figueiredo
Recent work has shown promising results in causal discovery by leveraging interventional data with gradient-based methods, even when the intervened variables are unknown. However,…
Modeling Structure with Undirected Neural Networks
Tsvetomila Mihaylova, Vlad Niculae, André F. T. Martins
Neural networks are powerful function estimators, leading to their status as a paradigm of choice for modeling structured data. However, unlike other structured representations tha…
SPECTRA: Sparse Structured Text Rationalization
Nuno Miguel Guerreiro, André F. T. Martins
Selective rationalization aims to produce decisions along with rationales (e.g., text highlights or word alignments between two sentences). Commonly, rationales are modeled as stoc…
Sparse Communication via Mixed Distributions
António Farinhas, Wilker Aziz, Vlad Niculae +1
Neural networks and other machine learning models compute continuous representations, while humans communicate mostly through discrete symbols. Reconciling these two forms of commu…
Sparse Continuous Distributions and Fenchel-Young Losses
André F. T. Martins, Marcos Treviso, António Farinhas +4
Exponential families are widely used in machine learning, including many distributions in continuous and discrete domains (e.g., Gaussian, Dirichlet, Poisson, and categorical distr…
Multimodal Continuous Visual Attention Mechanisms
António Farinhas, André F. T. Martins, Pedro M. Q. Aguiar
Visual attention mechanisms are a key component of neural network models for computer vision. By focusing on a discrete set of objects or image regions, these mechanisms identify t…
Reconciling the Discrete-Continuous Divide: Towards a Mathematical Theory of Sparse Communication
André F. T. Martins
Neural networks and other machine learning models compute continuous representations, while humans communicate with discrete symbols. Reconciling these two forms of communication i…
Understanding the Mechanics of SPIGOT: Surrogate Gradients for Latent Structure Learning
Tsvetomila Mihaylova, Vlad Niculae, André F. T. Martins
Latent structure models are a powerful tool for modeling language data: they can mitigate the error propagation and annotation bottleneck in pipeline systems, while simultaneously…
Efficient Marginalization of Discrete and Structured Latent Variables via Sparsity
Gonçalo M. Correia, Vlad Niculae, Wilker Aziz +1
Training neural network models with discrete (categorical or structured) latent variables can be computationally challenging, due to the need for marginalization over large or comb…