399 citations · 4k across the 100 of their papers we have counts for
34 papers · 1 filter
RegionCLIP: Region-based Language-Image Pretraining
Yiwu Zhong, Jianwei Yang, Pengchuan Zhang +8
Contrastive language-image pretraining (CLIP) using image-text pairs has achieved impressive results on image classification in both zero-shot and transfer learning settings. Howev…
Fine-Tuning Large Neural Language Models for Biomedical Natural Language Processing
Robert Tinn, Hao Cheng, Yu Gu +5
Motivation: A perennial challenge for biomedical researchers and clinical practitioners is to stay abreast with the rapid growth of publications and medical notes. Natural language…
ValueNet: A New Dataset for Human Value Driven Dialogue System
Liang Qiu, Yizhou Zhao, Jinchao Li +4
Building a socially intelligent agent involves many challenges, one of which is to teach the agent to speak guided by its value like a human. However, value-driven chatbots are sti…
KAT: A Knowledge Augmented Transformer for Vision-and-Language
Liangke Gui, Borui Wang, Qiuyuan Huang +3
The primary focus of recent work with largescale transformers has been on optimizing the amount of information packed into the model's parameters. In this work, we ask a different…
Knowledge-Grounded Dialogue Generation with a Unified Knowledge Representation
Yu Li, Baolin Peng, Yelong Shen +4
Knowledge-grounded dialogue systems are challenging to build due to the lack of training data and heterogeneous knowledge sources. Existing systems perform poorly on unseen topics…
Knowledge-Rich Self-Supervision for Biomedical Entity Linking
Sheng Zhang, Hao Cheng, Shikhar Vashishth +6
Entity linking faces significant challenges such as prolific variations and prevalent ambiguities, especially in high-value domains with myriad entities. Standard classification ap…
Grounded Language-Image Pre-training
Liunian Harold Li, Pengchuan Zhang, Haotian Zhang +9
This paper presents a grounded language-image pre-training (GLIP) model for learning object-level, language-aware, and semantic-rich visual representations. GLIP unifies object det…
Human Parity on CommonsenseQA: Augmenting Self-Attention with External Attention
Yichong Xu, Chenguang Zhu, Shuohang Wang +7
Most of today's AI systems focus on using self-attention mechanisms and transformer architectures on large amounts of diverse data to achieve impressive performance gains. In this…
Florence: A New Foundation Model for Computer Vision
Lu Yuan, Dongdong Chen, Yi-Ling Chen +20
Automated visual understanding of our diverse and open world demands computer vision models to generalize well with minimal customization for specific tasks, similar to human visio…
How much do language models copy from their training data? Evaluating linguistic novelty in text generation using RAVEN
R. Thomas McCoy, Paul Smolensky, Tal Linzen +2
Current language models can generate high-quality text. Are they simply copying text they have seen before, or have they learned generalizable linguistic abstractions? To tease apa…
CLUES: Few-Shot Learning Evaluation in Natural Language Understanding
Subhabrata Mukherjee, Xiaodong Liu, Guoqing Zheng +6
Most recent progress in natural language understanding (NLU) has been driven, in part, by benchmarks such as GLUE, SuperGLUE, SQuAD, etc. In fact, many NLU models have now matched…
DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing
Pengcheng He, Jianfeng Gao, Weizhu Chen
This paper presents a new pre-trained language model, DeBERTaV3, which improves the original DeBERTa model by replacing mask language modeling (MLM) with replaced token detection (…
Adversarial GLUE: A Multi-Task Benchmark for Robustness Evaluation of Language Models
Boxin Wang, Chejian Xu, Shuohang Wang +5
Large-scale pre-trained language models have achieved tremendous success across a wide range of natural language understanding (NLU) tasks, even surpassing human performance. Howev…
SYNERGY: Building Task Bots at Scale Using Symbolic Knowledge and Machine Teaching
Baolin Peng, Chunyuan Li, Zhu Zhang +3
In this paper we explore the use of symbolic knowledge and machine teaching to reduce human data labeling efforts in building neural task bots. We propose SYNERGY, a hybrid learnin…
Open Domain Question Answering with A Unified Knowledge Interface
Kaixin Ma, Hao Cheng, Xiaodong Liu +2
The retriever-reader framework is popular for open-domain question answering (ODQA) due to its ability to use explicit knowledge. Although prior work has sought to increase the kno…
LiST: Lite Prompted Self-training Makes Parameter-Efficient Few-shot Learners
Yaqing Wang, Subhabrata Mukherjee, Xiaodong Liu +3
We present a new method LiST is short for Lite Prompted Self-Training for parameter-efficient fine-tuning of large pre-trained language models (PLMs) for few-shot learning. LiST im…
Taming Sparsely Activated Transformer with Stochastic Experts
Simiao Zuo, Xiaodong Liu, Jian Jiao +5
Sparsely activated models (SAMs), such as Mixture-of-Experts (MoE), can easily scale to have outrageously large amounts of parameters without significant increase in computational…
ARCH: Efficient Adversarial Regularized Training with Caching
Simiao Zuo, Chen Liang, Haoming Jiang +5
Adversarial regularization can improve model generalization in many natural language processing tasks. However, conventional approaches are computationally expensive since they nee…
WebQA: Multihop and Multimodal QA
Yingshan Chang, Mridu Narang, Hisami Suzuki +3
Scaling Visual Question Answering (VQA) to the open-domain and multi-hop nature of web searches, requires fundamental advances in visual representation learning, knowledge aggregat…
TACo: Token-aware Cascade Contrastive Learning for Video-Text Alignment
Jianwei Yang, Yonatan Bisk, Jianfeng Gao
Contrastive learning has been widely used to train transformer-based vision-language models for video-text alignment and multi-modal representation learning. This paper presents a…
EmailSum: Abstractive Email Thread Summarization
Shiyue Zhang, Asli Celikyilmaz, Jianfeng Gao +1
Recent years have brought about an interest in the challenging task of summarizing conversation threads (meetings, online discussions, etc.). Such summaries help analysis of the lo…
Image Scene Graph Generation (SGG) Benchmark
Xiaotian Han, Jianwei Yang, Houdong Hu +3
There is a surge of interest in image scene graph generation (object, attribute and relationship detection) due to the need of building fine-grained image understanding models that…
Focal Self-attention for Local-Global Interactions in Vision Transformers
Jianwei Yang, Chunyuan Li, Pengchuan Zhang +4
Recently, Vision Transformer and its variants have shown great promise on various computer vision tasks. The ability of capturing short- and long-range visual dependencies through…
XtremeDistilTransformers: Task Transfer for Task-agnostic Distillation
Subhabrata Mukherjee, Ahmed Hassan Awadallah, Jianfeng Gao
While deep and large pre-trained models are the state-of-the-art for various natural language processing tasks, their huge size poses significant challenges for practical uses in r…