565 citations · 846 across the 18 of their papers we have counts for
5 papers · 1 filter
Do Attention Heads in BERT Track Syntactic Dependencies?
Phu Mon Htut, Jason Phang, Shikha Bordia +1
We investigate the extent to which individual attention heads in pretrained transformer language models, such as BERT and RoBERTa, implicitly capture syntactic dependency relations…
Generalized Inner Loop Meta-Learning
Edward Grefenstette, Brandon Amos, Denis Yarats +6
Many (but not all) approaches self-qualifying as "meta-learning" in deep learning and reinforcement learning fit a common pattern of approximating the solution to a nested optimiza…
Inducing Constituency Trees through Neural Machine Translation
Phu Mon Htut, Kyunghyun Cho, Samuel R. Bowman
Latent tree learning(LTL) methods learn to parse sentences using only indirect supervision from a downstream task. Recent advances in latent tree learning have made it possible to…
Investigating BERT's Knowledge of Language: Five Analysis Methods with NPIs
Alex Warstadt, Yu Cao, Ioana Grosu +13
Though state-of-the-art sentence representation models can perform tasks requiring significant knowledge of grammar, it is an open question how best to evaluate their grammatical k…
The Unbearable Weight of Generating Artificial Errors for Grammatical Error Correction
Phu Mon Htut, Joel Tetreault
In recent years, sequence-to-sequence models have been very effective for end-to-end grammatical error correction (GEC). As creating human-annotated parallel corpus for GEC is expe…