151 citations · 736 across the 85 of their papers we have counts for
27 papers · 1 filter
ECONET: Effective Continual Pretraining of Language Models for Event Temporal Reasoning
Rujun Han, Xiang Ren, Nanyun Peng
While pre-trained language models (PTLMs) have achieved noticeable success on many NLP tasks, they still struggle for tasks that require event temporal reasoning, which is essentia…
Pre-training Text-to-Text Transformers for Concept-centric Common Sense
Wangchunshu Zhou, Dong-Ho Lee, Ravi Kiran Selvam +3
Pre-trained language models (PTLM) have achieved impressive results in a range of natural language understanding (NLU) and generation (NLG) tasks. However, current pre-training obj…
Fair Hate Speech Detection through Evaluation of Social Group Counterfactuals
Aida Mostafazadeh Davani, Ali Omrani, Brendan Kennedy +3
Approaches for mitigating bias in supervised models are designed to reduce models' dependence on specific sensitive features of the input data, e.g., mentioned social groups. Howev…
One-shot Learning for Temporal Knowledge Graphs
Mehrnoosh Mirtaheri, Mohammad Rostami, Xiang Ren +2
Most real-world knowledge graphs are characterized by a long-tail relation frequency distribution where a significant fraction of relations occurs only a handful of times. This obs…
Will This Idea Spread Beyond Academia? Understanding Knowledge Transfer of Scientific Concepts across Text Corpora
Hancheng Cao, Mengjie Cheng, Zhepeng Cen +2
What kind of basic research ideas are more likely to get applied in practice? There is a long line of research investigating patterns of knowledge transfer, but it generally focuse…
Differentiable Open-Ended Commonsense Reasoning
Bill Yuchen Lin, Haitian Sun, Bhuwan Dhingra +3
Current commonsense reasoning research focuses on developing models that use commonsense knowledge to answer multiple-choice questions. However, systems designed to answer multiple…
Learning Contextualized Knowledge Structures for Commonsense Reasoning
Jun Yan, Mrigank Raman, Aaron Chan +6
Recently, knowledge graph (KG) augmented models have achieved noteworthy success on various commonsense reasoning tasks. However, KG edge (fact) sparsity and noisy edge extraction/…
Learning to Deceive Knowledge Graph Augmented Models via Targeted Perturbation
Mrigank Raman, Aaron Chan, Siddhant Agarwal +7
Knowledge graphs (KGs) have helped neural models improve performance on various knowledge-intensive tasks, like question answering and item recommendation. By using attention over…
On Transferability of Bias Mitigation Effects in Language Model Fine-Tuning
Xisen Jin, Francesco Barbieri, Brendan Kennedy +3
Fine-tuned language models have been shown to exhibit biases against protected groups in a host of modeling tasks such as text classification and coreference resolution. Previous w…
Constrained Abstractive Summarization: Preserving Factual Consistency with Constrained Generation
Yuning Mao, Xiang Ren, Heng Ji +1
Despite significant progress, state-of-the-art abstractive summarization methods are still prone to hallucinate content inconsistent with the source document. In this paper, we pro…
Semi-Automated Protocol Disambiguation and Code Generation
Jane Yen, Tamás Lévai, Qinyuan Ye +3
For decades, Internet protocols have been specified using natural language. Given the ambiguity inherent in such text, it is not surprising that protocol implementations have long…
Multi-document Summarization with Maximal Marginal Relevance-guided Reinforcement Learning
Yuning Mao, Yanru Qu, Yiqing Xie +2
While neural sequence learning methods have made significant progress in single-document summarization (SDS), they produce unsatisfactory results on multi-document summarization (M…
SynSetExpan: An Iterative Framework for Joint Entity Set Expansion and Synonym Discovery
Jiaming Shen, Wenda Qiu, Jingbo Shang +3
Entity set expansion and synonym discovery are two critical NLP tasks. Previous studies accomplish them separately, without exploring their interdependencies. In this work, we hypo…
Gradient-based Editing of Memory Examples for Online Task-free Continual Learning
Xisen Jin, Arka Sadhu, Junyi Du +1
We explore task-free continual learning (CL), in which a model is trained to avoid catastrophic forgetting in the absence of explicit task boundaries or identities. Among many effo…
Screenplay Quality Assessment: Can We Predict Who Gets Nominated?
Ming-Chang Chiu, Tiantian Feng, Xiang Ren +1
Deciding which scripts to turn into movies is a costly and time-consuming process for filmmakers. Thus, building a tool to aid script selection, an initial phase in movie productio…
Contextualizing Hate Speech Classifiers with Post-hoc Explanation
Brendan Kennedy, Xisen Jin, Aida Mostafazadeh Davani +2
Hate speech classifiers trained on imbalanced datasets struggle to determine if group identifiers like "gay" or "black" are used in offensive or prejudiced ways. Such biases manife…
IsoBN: Fine-Tuning BERT with Isotropic Batch Normalization
Wenxuan Zhou, Bill Yuchen Lin, Xiang Ren
Fine-tuning pre-trained language models (PTLMs), such as BERT and its better variant RoBERTa, has been a common practice for advancing performance in natural language understanding…
Teaching Machine Comprehension with Compositional Explanations
Qinyuan Ye, Xiao Huang, Elizabeth Boschee +1
Advances in machine reading comprehension (MRC) rely heavily on the collection of large scale human-annotated examples in the form of (question, paragraph, answer) triples. In cont…
ForecastQA: A Question Answering Challenge for Event Forecasting with Temporal Text Data
Woojeong Jin, Rahul Khanna, Suji Kim +4
Event forecasting is a challenging, yet important task, as humans seek to constantly plan for the future. Existing automated forecasting studies rely mostly on structured data, suc…
Visually Grounded Continual Learning of Compositional Phrases
Xisen Jin, Junyi Du, Arka Sadhu +2
Humans acquire language continually with much more limited access to data samples at a time, as compared to contemporary NLP systems. To study this human-like language acquisition…
RICA: Evaluating Robust Inference Capabilities Based on Commonsense Axioms
Pei Zhou, Rahul Khanna, Seyeon Lee +4
Pre-trained language models (PTLMs) have achieved impressive performance on commonsense inference benchmarks, but their ability to employ commonsense to make robust inferences, whi…
Connecting the Dots: A Knowledgeable Path Generator for Commonsense Question Answering
Peifeng Wang, Nanyun Peng, Filip Ilievski +2
Commonsense question answering (QA) requires background knowledge which is not explicitly stated in a given context. Prior works use commonsense knowledge graphs (KGs) to obtain th…
Birds have four legs?! NumerSense: Probing Numerical Commonsense Knowledge of Pre-trained Language Models
Bill Yuchen Lin, Seyeon Lee, Rahul Khanna +1
Recent works show that pre-trained language models (PTLMs), such as BERT, possess certain commonsense and factual knowledge. They suggest that it is promising to use PTLMs as "neur…
Scalable Multi-Hop Relational Reasoning for Knowledge-Aware Question Answering
Yanlin Feng, Xinyue Chen, Bill Yuchen Lin +3
Existing work on augmenting question answering (QA) models with external knowledge (e.g., knowledge graphs) either struggle to model multi-hop relations efficiently, or lack transp…