2.2k citations · 2.5k across the 23 of their papers we have counts for
6 papers · 1 filter
Efficient Hierarchical Domain Adaptation for Pretrained Language Models
Alexandra Chronopoulou, Matthew E. Peters, Jesse Dodge
The remarkable success of large language models has been driven by dense models trained on massive unlabeled, unstructured corpora. These corpora typically contain text from divers…
Few-Shot Self-Rationalization with Natural Language Prompts
Ana Marasović, Iz Beltagy, Doug Downey +1
Self-rationalization models that predict task labels and generate free-text elaborations for their predictions could enable more intuitive interaction with NLP systems. These model…
Tailor: Generating and Perturbing Text with Semantic Controls
Alexis Ross, Tongshuang Wu, Hao Peng +2
Controlled text perturbation is useful for evaluating and improving model generalizability. However, current techniques rely on training a model for every target perturbation, whic…
PIGLeT: Language Grounding Through Neuro-Symbolic Interaction in a 3D World
Rowan Zellers, Ari Holtzman, Matthew Peters +4
We propose PIGLeT: a model that learns physical commonsense knowledge through interaction, and then uses this knowledge to ground language. We factorize PIGLeT into a physical dyna…
Competency Problems: On Finding and Removing Artifacts in Language Data
Matt Gardner, William Merrill, Jesse Dodge +4
Much recent work in NLP has documented dataset artifacts, bias, and spurious correlations between input features and output labels. However, how to tell which features have "spurio…
CDLM: Cross-Document Language Modeling
Avi Caciularu, Arman Cohan, Iz Beltagy +3
We introduce a new pretraining approach geared for multi-document language modeling, incorporating two key ideas into the masked language modeling self-supervised objective. First,…