1.3k citations · 2.8k across the 31 of their papers we have counts for
32 papers
Mobile V-MoEs: Scaling Down Vision Transformers via Sparse Mixture-of-Experts
Erik Daxberger, Floris Weers, Bowen Zhang +7
Sparse Mixture-of-Experts models (MoEs) have recently gained popularity due to their ability to decouple model size from inference efficiency by only activating a small subset of t…
MOFI: Learning Image Representations from Noisy Entity Annotated Images
Wentao Wu, Aleksei Timofeev, Chen Chen +8
We present MOFI, Manifold OF Images, a new vision foundation model designed to learn image representations from noisy entity annotated images. MOFI differs from previous work in tw…
STAIR: Learning Sparse Text and Image Representation in Grounded Tokens
Chen Chen, Bowen Zhang, Liangliang Cao +7
Image and text retrieval is one of the foundational tasks in the vision and language domain with multiple real-world applications. State-of-the-art approaches, e.g. CLIP, ALIGN, re…
Masked Autoencoding Does Not Help Natural Language Supervision at Scale
Floris Weers, Vaishaal Shankar, Angelos Katharopoulos +2
Self supervision and natural language supervision have emerged as two exciting ways to train general purpose image encoders which excel at a variety of downstream tasks. Recent wor…
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision
Chao Jia, Yinfei Yang, Ye Xia +7
Pre-trained representations are becoming crucial for many NLP and perception tasks. While representation learning in NLP has transitioned to training on raw text without human anno…
Cross-Modal Contrastive Learning for Text-to-Image Generation
Han Zhang, Jing Yu Koh, Jason Baldridge +2
The output of text-to-image synthesis systems should be coherent, clear, photo-realistic scenes with high semantic fidelity to their conditioned text descriptions. Our Cross-Modal…
Universal Sentence Representation Learning with Conditional Masked Language Model
Ziyi Yang, Yinfei Yang, Daniel Cer +2
This paper presents a novel training method, Conditional Masked Language Modeling (CMLM), to effectively learn sentence representations on large scale unlabeled corpora. CMLM integ…
Text-to-Image Generation Grounded by Fine-Grained User Attention
Jing Yu Koh, Jason Baldridge, Honglak Lee +1
Localized Narratives is a dataset with detailed natural language descriptions of images paired with mouse traces that provide a sparse, fine-grained visual grounding for phrases. W…
Neural Passage Retrieval with Improved Negative Contrast
Jing Lu, Gustavo Hernandez Abrego, Ji Ma +2
In this paper we explore the effects of negative sampling in dual encoder models used to retrieve passages for automatic question answering. We explore four negative sampling strat…
Neural Retrieval for Question Answering with Cross-Attention Supervised Data Augmentation
Yinfei Yang, Ning Jin, Kuo Lin +2
Neural models that independently project questions and answers into a shared embedding space allow for efficient continuous space retrieval from large corpora. Independently comput…
MultiReQA: A Cross-Domain Evaluation for Retrieval Question Answering Models
Mandy Guo, Yinfei Yang, Daniel Cer +2
Retrieval question answering (ReQA) is the task of retrieving a sentence-level answer to a question from an open corpus (Ahmad et al.,2019).This paper presents MultiReQA, anew mult…
SueNes: A Weakly Supervised Approach to Evaluating Single-Document Summarization via Negative Sampling
Forrest Sheng Bao, Hebi Li, Ge Luo +4
Canonical automatic summary evaluation metrics, such as ROUGE, focus on lexical similarity which cannot well capture semantics nor linguistic quality and require a reference summar…
LAReQA: Language-agnostic answer retrieval from a multilingual pool
Uma Roy, Noah Constant, Rami Al-Rfou +3
We present LAReQA, a challenging new benchmark for language-agnostic answer retrieval from a multilingual candidate pool. Unlike previous cross-lingual tasks, LAReQA tests for "str…
Crisscrossed Captions: Extended Intramodal and Intermodal Semantic Similarity Judgments for MS-COCO
Zarana Parekh, Jason Baldridge, Daniel Cer +2
By supporting multi-modal retrieval training and evaluation, image captioning datasets have spurred remarkable progress on representation learning. Unfortunately, datasets have lim…
Interpretability Analysis for Named Entity Recognition to Understand System Predictions and How They Can Improve
Oshin Agarwal, Yinfei Yang, Byron C. Wallace +1
Named Entity Recognition systems achieve remarkable performance on domains such as English news. It is natural to ask: What are these models actually learning to achieve this? Are…
Entity-Switched Datasets: An Approach to Auditing the In-Domain Robustness of Named Entity Recognition Models
Oshin Agarwal, Yinfei Yang, Byron C. Wallace +1
Named entity recognition systems perform well on standard datasets comprising English news. But given the paucity of data, it is difficult to draw conclusions about the robustness…
PAWS-X: A Cross-lingual Adversarial Dataset for Paraphrase Identification
Yinfei Yang, Yuan Zhang, Chris Tar +1
Most existing work on adversarial data generation focuses on English. For example, PAWS (Paraphrase Adversaries from Word Scrambling) consists of challenging English paraphrase ide…
Learning a Multi-Domain Curriculum for Neural Machine Translation
Wei Wang, Ye Tian, Jiquan Ngiam +3
Most data selection research in machine translation focuses on improving a single domain. We perform data selection for multiple domains at once. This is achieved by carefully intr…
Multilingual Universal Sentence Encoder for Semantic Retrieval
Yinfei Yang, Daniel Cer, Amin Ahmad +9
We introduce two pre-trained retrieval focused multilingual sentence encoding models, respectively based on the Transformer and CNN model architectures. The models embed text from…
ReQA: An Evaluation for End-to-End Answer Retrieval Models
Amin Ahmad, Noah Constant, Yinfei Yang +1
Popular QA benchmarks like SQuAD have driven progress on the task of identifying answer spans within a specific passage, with models now surpassing human performance. However, retr…
Hierarchical Document Encoder for Parallel Corpus Mining
Mandy Guo, Yinfei Yang, Keith Stevens +5
We explore using multilingual document embeddings for nearest neighbor mining of parallel data. Three document-level representations are investigated: (i) document embeddings gener…
Predicting Annotation Difficulty to Improve Task Routing and Model Performance for Biomedical Information Extraction
Yinfei Yang, Oshin Agarwal, Chris Tar +2
Modern NLP systems require high-quality annotated data. In specialized domains, expert annotations may be prohibitively expensive. An alternative is to rely on crowdsourcing to red…
Improving Multilingual Sentence Embedding using Bi-directional Dual Encoder with Additive Margin Softmax
Yinfei Yang, Gustavo Hernandez Abrego, Steve Yuan +6
In this paper, we present an approach to learn multilingual sentence embeddings using a bi-directional dual-encoder with additive margin softmax. The embeddings are able to achieve…
Learning Cross-Lingual Sentence Representations via a Multi-task Dual-Encoder Model
Muthuraman Chidambaram, Yinfei Yang, Daniel Cer +4
A significant roadblock in multilingual neural language modeling is the lack of labeled non-English data. One potential method for overcoming this issue is learning cross-lingual t…