463 citations · 5.1k across the 190 of their papers we have counts for
30 papers · 1 filter
Pseudo-OOD training for robust language models
Dhanasekar Sundararaman, Nikhil Mehta, Lawrence Carin
While pre-trained large-scale deep models have garnered attention as an important topic for many downstream natural language processing (NLP) tasks, such models often make unreliab…
Improving Downstream Task Performance by Treating Numbers as Entities
Dhanasekar Sundararaman, Vivek Subramanian, Guoyin Wang +2
Numbers are essential components of text, like any other word tokens, from which natural language processing (NLP) models are built and deployed. Though numbers are typically not a…
elBERto: Self-supervised Commonsense Learning for Question Answering
Xunlin Zhan, Yuan Li, Xiao Dong +3
Commonsense question answering requires reasoning about everyday situations and causes and effects implicit in context. Typically, existing approaches first retrieve external evide…
FairFil: Contrastive Neural Debiasing Method for Pretrained Text Encoders
Pengyu Cheng, Weituo Hao, Siyang Yuan +2
Pretrained text encoders, such as BERT, have been applied increasingly in various natural language processing (NLP) tasks, and have recently demonstrated significant performance ga…
What Makes Good In-Context Examples for GPT-?
Jiachang Liu, Dinghan Shen, Yizhe Zhang +3
GPT- has attracted lots of attention due to its superior performance across a wide range of NLP tasks, especially with its powerful and versatile in-context few-shot learning ab…
MixKD: Towards Efficient Distillation of Large-scale Language Models
Kevin J Liang, Weituo Hao, Dinghan Shen +4
Large-scale language models have recently demonstrated impressive empirical performance. Nevertheless, the improved results are attained at the price of bigger models, more power c…
Improving Text Generation with Student-Forcing Optimal Transport
Guoyin Wang, Chunyuan Li, Jianqiao Li +10
Neural language models are often trained with maximum likelihood estimation (MLE), where the next word is generated conditioned on the ground-truth word tokens. During testing, how…
Graph Optimal Transport for Cross-Domain Alignment
Liqun Chen, Zhe Gan, Yu Cheng +3
Cross-domain alignment between two sets of entities (e.g., objects in an image, words in a sentence) is fundamental to both computer vision and natural language processing. Existin…
Students Need More Attention: BERT-based AttentionModel for Small Data with Application to AutomaticPatient Message Triage
Shijing Si, Rui Wang, Jedrek Wosik +5
Small and imbalanced datasets commonly seen in healthcare represent a challenge when training classifiers based on deep learning models. So motivated, we propose a novel framework…
Reward Constrained Interactive Recommendation with Natural Language Feedback
Ruiyi Zhang, Tong Yu, Yilin Shen +3
Text-based interactive recommendation provides richer user feedback and has demonstrated advantages over traditional interactive recommender systems. However, recommendations can e…
Improving Adversarial Text Generation by Modeling the Distant Future
Ruiyi Zhang, Changyou Chen, Zhe Gan +5
Auto-regressive text generation models usually focus on local fluency, and may cause inconsistent semantic meaning in long text generation. Further, automatically generating words…
Nested-Wasserstein Self-Imitation Learning for Sequence Generation
Ruiyi Zhang, Changyou Chen, Zhe Gan +3
Reinforcement learning (RL) has been widely studied for improving sequence-generation models. However, the conventional rewards used for RL training typically cannot capture suffic…
Syntax-Infused Transformer and BERT models for Machine Translation and Natural Language Understanding
Dhanasekar Sundararaman, Vivek Subramanian, Guoyin Wang +4
Attention-based models have shown significant improvement over traditional algorithms in several NLP tasks. The Transformer, for instance, is an illustrative example that generates…
Ouroboros: On Accelerating Training of Transformer-Based Language Models
Qian Yang, Zhouyuan Huo, Wenlin Wang +2
Language models are essential for natural language processing (NLP) tasks, such as machine translation and text summarization. Remarkable performance has been demonstrated recently…
Learning Compressed Sentence Representations for On-Device Text Processing
Dinghan Shen, Pengyu Cheng, Dhanasekar Sundararaman +5
Vector representations of sentences, trained on massive text corpora, are widely used as generic sentence embeddings across a variety of NLP problems. The learned representations a…
Improving Textual Network Embedding with Global Attention via Optimal Transport
Liqun Chen, Guoyin Wang, Chenyang Tao +6
Constituting highly informative network embeddings is an important tool for network analysis. It encodes network topology, along with other useful side information, into low-dimens…
Topic-Guided Variational Autoencoders for Text Generation
Wenlin Wang, Zhe Gan, Hongteng Xu +5
We propose a topic-guided variational autoencoder (TGVAE) model for text generation. Distinct from existing variational autoencoder (VAE) based approaches, which assume a simple Ga…
Towards Generating Long and Coherent Text with Multi-Level Latent Variable Models
Dinghan Shen, Asli Celikyilmaz, Yizhe Zhang +4
Variational autoencoders (VAEs) have received much attention recently as an end-to-end architecture for text generation with latent variables. In this paper, we investigate several…
Improving Sequence-to-Sequence Learning via Optimal Transport
Liqun Chen, Yizhe Zhang, Ruiyi Zhang +7
Sequence-to-sequence models are commonly trained via maximum likelihood estimation (MLE). However, standard MLE training considers a word-level objective, predicting the next word…
Sequence Generation with Guider Network
Ruiyi Zhang, Changyou Chen, Zhe Gan +5
Sequence generation with reinforcement learning (RL) has received significant attention recently. However, a challenge with such methods is the sparse-reward problem in the RL trai…
Adversarial Text Generation via Feature-Mover's Distance
Liqun Chen, Shuyang Dai, Chenyang Tao +5
Generative adversarial networks (GANs) have achieved significant success in generating real-valued data. However, the discrete nature of text hinders the application of GAN to text…
Improved Semantic-Aware Network Embedding with Fine-Grained Word Alignment
Dinghan Shen, Xinyuan Zhang, Ricardo Henao +1
Network embeddings, which learn low-dimensional representations for each vertex in a large-scale network, have received considerable attention in recent years. For a wide range of…
Baseline Needs More Love: On Simple Word-Embedding-Based Models and Associated Pooling Mechanisms
Dinghan Shen, Guoyin Wang, Wenlin Wang +6
Many deep learning architectures have been proposed to model the compositionality in text sequences, requiring a substantial number of parameters and expensive computations. Howeve…
NASH: Toward End-to-End Neural Architecture for Generative Semantic Hashing
Dinghan Shen, Qinliang Su, Paidamoyo Chapfuwa +4
Semantic hashing has become a powerful paradigm for fast similarity search in many information retrieval systems. While fairly successful, previous techniques generally require two…