93 citations · 446 across the 32 of their papers we have counts for
27 papers · 1 filter
Can You Learn to See Without Images? Procedural Warm-Up for Vision Transformers
Zachary Shinnick, Liangze Jiang, Hemanth Saratchandran +2
Transformers are remarkably versatile, suggesting the existence of generic inductive biases beneficial across modalities. In this work, we explore a new way to instil such biases i…
Synergy and Diversity in CLIP: Enhancing Performance Through Adaptive Backbone Ensembling
Cristian Rodriguez-Opazo, Ehsan Abbasnejad, Damien Teney +3
Contrastive Language-Image Pretraining (CLIP) stands out as a prominent method for image representation learning. Various architectures, from vision transformers (ViTs) to convolut…
Zero-shot Retrieval: Augmenting Pre-trained Models with Search Engines
Hamed Damirchi, Cristian Rodríguez-Opazo, Ehsan Abbasnejad +4
Large pre-trained models can dramatically reduce the amount of task-specific data required to solve a problem, but they often fail to capture domain-specific nuances out of the box…
Learning Diverse Features in Vision Transformers for Improved Generalization
Armand Mihai Nicolicioiu, Andrei Liviu Nicolicioiu, Bogdan Alexe +1
Deep learning models often rely only on a small set of features even when there is a rich set of predictive signals in the training data. This makes models brittle and sensitive to…
Candidate Set Re-ranking for Composed Image Retrieval with Dual Multi-modal Encoder
Zheyuan Liu, Weixuan Sun, Damien Teney +1
Composed image retrieval aims to find an image that best matches a given multi-modal user query consisting of a reference image and text pair. Existing methods commonly pre-compute…
Bi-directional Training for Composed Image Retrieval via Text Prompt Learning
Zheyuan Liu, Weixuan Sun, Yicong Hong +2
Composed image retrieval searches for a target image based on a multi-modal user query comprised of a reference image and modification text describing the desired changes. Existing…
SelecMix: Debiased Learning by Contradicting-pair Sampling
Inwoo Hwang, Sangjun Lee, Yunhyeok Kwak +4
Neural networks trained with ERM (empirical risk minimization) sometimes learn unintended decision rules, in particular when their training data is biased, i.e., when training labe…
EBMs vs. CL: Exploring Self-Supervised Visual Pretraining for Visual Question Answering
Violetta Shevchenko, Ehsan Abbasnejad, Anthony Dick +2
The availability of clean and diverse labeled data is a major roadblock for training models on complex tasks such as visual question answering (VQA). The extensive work on large vi…
Active Learning by Feature Mixing
Amin Parvaneh, Ehsan Abbasnejad, Damien Teney +3
The promise of active learning (AL) is to reduce labelling costs by selecting the most valuable examples to annotate from a pool of unlabelled data. Identifying these examples is e…
Image Retrieval on Real-life Images with Pre-trained Vision-and-Language Models
Zheyuan Liu, Cristian Rodriguez-Opazo, Damien Teney +1
We extend the task of composed image retrieval, where an input query consists of an image and short textual description of how to modify the image. Existing methods have only been…
Beyond Question-Based Biases: Assessing Multimodal Shortcut Learning in Visual Question Answering
Corentin Dancette, Remi Cadene, Damien Teney +1
We introduce an evaluation methodology for visual question answering (VQA) to better diagnose cases of shortcut learning. These cases happen when a model exploits spurious statisti…
Reasoning over Vision and Language: Exploring the Benefits of Supplemental Knowledge
Violetta Shevchenko, Damien Teney, Anthony Dick +1
The limits of applicability of vision-and-language models are defined by the coverage of their training data. Tasks like vision question answering (VQA) often require commonsense a…
On the Value of Out-of-Distribution Testing: An Example of Goodhart's Law
Damien Teney, Kushal Kafle, Robik Shrestha +3
Out-of-distribution (OOD) testing is increasingly popular for evaluating a machine learning system's ability to generalize beyond the biases of a training set. OOD benchmarks are d…
Visual Question Answering with Prior Class Semantics
Violetta Shevchenko, Damien Teney, Anthony Dick +1
We present a novel mechanism to embed prior knowledge in a model for visual question answering. The open-set nature of the task is at odds with the ubiquitous approach of training…
Learning What Makes a Difference from Counterfactual Examples and Gradient Supervision
Damien Teney, Ehsan Abbasnedjad, Anton van den Hengel
One of the primary challenges limiting the applicability of deep learning is its susceptibility to learning spurious correlations rather than the underlying mechanisms of the task…
Unshuffling Data for Improved Generalization
Damien Teney, Ehsan Abbasnejad, Anton van den Hengel
Generalization beyond the training distribution is a core challenge in machine learning. The common practice of mixing and shuffling examples when training neural networks may not…
On Incorporating Semantic Prior Knowledge in Deep Learning Through Embedding-Space Constraints
Damien Teney, Ehsan Abbasnejad, Anton van den Hengel
The knowledge that humans hold about a problem often extends far beyond a set of training data and output labels. While the success of deep learning mostly relies on supervised tra…
V-PROM: A Benchmark for Visual Reasoning Using Visual Progressive Matrices
Damien Teney, Peng Wang, Jiewei Cao +3
One of the primary challenges faced by deep learning is the degree to which current methods exploit superficial statistics and dataset bias, rather than learning to generalise over…
Actively Seeking and Learning from Live Data
Damien Teney, Anton van den Hengel
One of the key limitations of traditional machine learning methods is their requirement for training data that exemplifies all the information to be learned. This is a particular p…
Visual Question Answering as a Meta Learning Task
Damien Teney, Anton van den Hengel
The predominant approach to Visual Question Answering (VQA) demands that the model represents within its weights all of the information required to answer any question about any im…
Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environments
Peter Anderson, Qi Wu, Damien Teney +6
A robot that can carry out a natural-language instruction has been a dream since before the Jetsons cartoon series imagined a life of leisure mediated by a fleet of attentive robot…
Tips and Tricks for Visual Question Answering: Learnings from the 2017 Challenge
Damien Teney, Peter Anderson, Xiaodong He +1
This paper presents a state-of-the-art model for visual question answering (VQA), which won the first place in the 2017 VQA Challenge. VQA is a task of significant importance for r…
Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering
Peter Anderson, Xiaodong He, Chris Buehler +4
Top-down visual attention mechanisms have been used extensively in image captioning and visual question answering (VQA) to enable deeper image understanding through fine-grained an…
Zero-Shot Visual Question Answering
Damien Teney, Anton van den Hengel
Part of the appeal of Visual Question Answering (VQA) is its promise to answer new questions about previously unseen images. Most current methods demand training questions that ill…