236 citations · 679 across the 32 of their papers we have counts for
36 papers
Contrastive Lift: 3D Object Instance Segmentation by Slow-Fast Contrastive Fusion
Yash Bhalgat, Iro Laina, João F. Henriques +2
Instance segmentation in 3D is a challenging task due to the lack of large-scale annotated datasets. In this paper, we show that this task can be addressed effectively by leveragin…
Extracting Reward Functions from Diffusion Models
Felipe Nuti, Tim Franzmeyer, João F. Henriques
Diffusion models have achieved remarkable results in image generation, and have similarly been used to learn high-performing policies in sequential decision-making tasks. Decision-…
Large Language Models are Few-shot Publication Scoopers
Samuel Albanie, Liliane Momeni, João F. Henriques
Driven by recent advances AI, we passengers are entering a golden age of scientific discovery. But golden for whom? Confronting our insecurity that others may beat us to the most a…
Towards Solving Fuzzy Tasks with Human Feedback: A Retrospective of the MineRL BASALT 2022 Competition
Stephanie Milani, Anssi Kanervisto, Karolis Ramanauskas +27
To facilitate research in the direction of fine-tuning foundation models from human feedback, we held the MineRL BASALT Competition on Fine-Tuning from Human Feedback at NeurIPS 20…
A Light Touch Approach to Teaching Transformers Multi-view Geometry
Yash Bhalgat, Joao F. Henriques, Andrew Zisserman
Transformers are powerful visual learners, in large part due to their conspicuous lack of manually-specified priors. This flexibility can be problematic in tasks that involve multi…
RbA: Segmenting Unknown Regions Rejected by All
Nazir Nayal, Mısra Yavuz, João F. Henriques +1
Standard semantic segmentation models owe their success to curated datasets with a fixed set of semantic categories, without contemplating the possibility of identifying unknown ob…
CASSPR: Cross Attention Single Scan Place Recognition
Yan Xia, Mariia Gladkova, Rui Wang +4
Place recognition based on point clouds (LiDAR) is an important component for autonomous robots or self-driving vehicles. Current SOTA performance is achieved on accumulated LiDAR…
Learn what matters: cross-domain imitation learning with task-relevant embeddings
Tim Franzmeyer, Philip H. S. Torr, João F. Henriques
We study how an autonomous agent learns to perform a task from demonstrations in a different domain, such as a different environment or different agent. Such cross-domain imitation…
SNeS: Learning Probably Symmetric Neural Surfaces from Incomplete Data
Eldar Insafutdinov, Dylan Campbell, João F. Henriques +1
We present a method for the accurate 3D reconstruction of partly-symmetric objects. We build on the strengths of recent advances in neural reconstruction and rendering such as Neur…
A 23 MW data centre is all you need
Samuel Albanie, Dylan Campbell, João F. Henriques
The field of machine learning has achieved striking progress in recent years, witnessing breakthrough results on language modelling, protein folding and nitpickingly fine-grained d…
Audio Retrieval with Natural Language Queries: A Benchmark Study
A. Sophia Koepke, Andreea-Maria Oncescu, João F. Henriques +2
The objectives of this work are cross-modal text-audio and audio-text retrieval, in which the goal is to retrieve the audio content from a pool of candidates that best matches a gi…
Quantised Transforming Auto-Encoders: Achieving Equivariance to Arbitrary Transformations in Deep Networks
Jianbo Jiao, João F. Henriques
In this work we investigate how to achieve equivariance to input transformations in deep networks, purely from data, without being given a model of those transformations. Convoluti…
Towards real-world navigation with deep differentiable planners
Shu Ishida, João F. Henriques
We train embodied neural networks to plan and navigate unseen complex 3D environments, emphasising real-world deployment. Rather than requiring prior knowledge of the agent or envi…
Learning Altruistic Behaviours in Reinforcement Learning without External Rewards
Tim Franzmeyer, Mateusz Malinowski, João F. Henriques
Can artificial agents learn to assist others in achieving their goals without knowing what those goals are? Generic reinforcement learning agents could be trained to behave altruis…
Keeping Your Eye on the Ball: Trajectory Attention in Video Transformers
Mandela Patrick, Dylan Campbell, Yuki M. Asano +5
In video transformers, the time dimension is often treated in the same way as the two spatial dimensions. However, in a scene where objects or the camera may move, a physical point…
Moving SLAM: Fully Unsupervised Deep Learning in Non-Rigid Scenes
Dan Xu, Andrea Vedaldi, Joao F. Henriques
We propose a method to train deep networks to decompose videos into 3D geometry (camera and depth), moving objects, and their motions, with no supervision. We build on the idea of…
Audio Retrieval with Natural Language Queries
Andreea-Maria Oncescu, A. Sophia Koepke, João F. Henriques +2
We consider the task of retrieving audio using free-form natural language queries. To study this problem, which has received limited attention in the existing literature, we introd…
On the Origin of Species of Self-Supervised Learning
Samuel Albanie, Erika Lu, Joao F. Henriques
In the quiet backwaters of cs.CV, cs.LG and stat.ML, a cornucopia of new learning systems is emerging from a primordial soup of mathematics-learning systems with no need for extern…
Space-Time Crop & Attend: Improving Cross-modal Video Representation Learning
Mandela Patrick, Yuki M. Asano, Bernie Huang +4
The quality of the image representations obtained from self-supervised learning depends strongly on the type of data augmentations used in the learning formulation. Recent papers h…
QuerYD: A video dataset with high-quality text and audio narrations
Andreea-Maria Oncescu, João F. Henriques, Yang Liu +2
We introduce QuerYD, a new large-scale dataset for retrieval and event localisation in video. A unique feature of our dataset is the availability of two audio tracks for each video…
Support-set bottlenecks for video-text representation learning
Mandela Patrick, Po-Yao Huang, Yuki Asano +4
The dominant paradigm for learning video-text representations -- noise contrastive learning -- increases the similarity of the representations of pairs of samples that are known to…
Automatic Recall Machines: Internal Replay, Continual Learning and the Brain
Xu Ji, Joao Henriques, Tinne Tuytelaars +1
Replay in neural networks involves training on sequential data with memorized samples, which counteracts forgetting of previous behavior caused by non-stationarity. We present a me…
State-of-Art-Reviewing: A Radical Proposal to Improve Scientific Publication
Samuel Albanie, Jaime Thewmore, Robert McCraith +1
Peer review forms the backbone of modern scientific manuscript evaluation. But after two hundred and eighty-nine years of egalitarian service to the scientific community, does this…
On Compositions of Transformations in Contrastive Self-Supervised Learning
Mandela Patrick, Yuki M. Asano, Polina Kuznetsova +4
In the image domain, excellent representations can be learned by inducing invariance to content-preserving transformations via noise contrastive learning. In this paper, we general…