13 citations · 14 across the 6 of their papers we have counts for
10 papers
Principled Gradient-based Markov Chain Monte Carlo for Text Generation
Li Du, Afra Amini, Lucas Torroba Hennigen +4
Recent papers have demonstrated the possibility of energy-based text generation by adapting gradient-based sampling algorithms, a paradigm of MCMC algorithms that promises fast con…
Deriving Language Models from Masked Language Models
Lucas Torroba Hennigen, Yoon Kim
Masked language models (MLM) do not explicitly define a distribution over language, i.e., they are not language models per se. However, recent work has implicitly treated them as s…
Learning to Grow Pretrained Models for Efficient Transformer Training
Peihao Wang, Rameswar Panda, Lucas Torroba Hennigen +6
Scaling transformers has led to significant breakthroughs in many domains, leading to a paradigm in which larger versions of existing models are trained and released on a periodic…
An Ordinal Latent Variable Model of Conflict Intensity
Niklas Stoehr, Lucas Torroba Hennigen, Josef Valvoda +3
Measuring the intensity of events is crucial for monitoring and tracking armed conflict. Advances in automated event extraction have yielded massive data sets of "who did what to w…
Same Neurons, Different Languages: Probing Morphosyntax in Multilingual Pre-trained Models
Karolina Stańczak, Edoardo Ponti, Lucas Torroba Hennigen +2
The success of multilingual pre-trained models is underpinned by their ability to learn representations shared by multiple languages even in absence of any explicit supervision. Ho…
A Latent-Variable Model for Intrinsic Probing
Karolina Stańczak, Lucas Torroba Hennigen, Adina Williams +2
The success of pre-trained contextualized representations has prompted researchers to analyze them for the presence of linguistic information. Indeed, it is natural to assume that…
Probing as Quantifying Inductive Bias
Alexander Immer, Lucas Torroba Hennigen, Vincent Fortuin +1
Pre-trained contextual representations have led to dramatic performance improvements on a range of downstream tasks. Such performance improvements have motivated researchers to qua…
Classifying Dyads for Militarized Conflict Analysis
Niklas Stoehr, Lucas Torroba Hennigen, Samin Ahbab +2
Understanding the origins of militarized conflict is a complex, yet important undertaking. Existing research seeks to build this understanding by considering bi-lateral relationshi…
Intrinsic Probing through Dimension Selection
Lucas Torroba Hennigen, Adina Williams, Ryan Cotterell
Most modern NLP systems make use of pre-trained contextual representations that attain astonishingly high performance on a variety of tasks. Such high performance should not be pos…
SIGMORPHON 2020 Shared Task 0: Typologically Diverse Morphological Inflection
Ekaterina Vylomova, Jennifer White, Elizabeth Salesky +25
A broad goal in natural language processing (NLP) is to develop a system that has the capacity to process any natural language. Most systems, however, are developed using data from…