503 citations · 1.3k across the 34 of their papers we have counts for
17 papers · 1 filter
Perception Test: A Diagnostic Benchmark for Multimodal Video Models
Viorica Pătrăucean, Lucas Smaira, Ankush Gupta +21
We propose a novel multimodal video benchmark - the Perception Test - to evaluate the perception and reasoning skills of pre-trained multimodal models (e.g. Flamingo, SeViLA, or GP…
Compressed Vision for Efficient Video Understanding
Olivia Wiles, Joao Carreira, Iain Barr +2
Experience and reasoning occur across multiple temporal scales: milliseconds, seconds, hours or days. The vast majority of computer vision research, however, still focuses on indiv…
CLIP-CLOP: CLIP-Guided Collage and Photomontage
Piotr Mirowski, Dylan Banarse, Mateusz Malinowski +2
The unabated mystique of large-scale neural networks, such as the CLIP dual image-and-text encoder, popularized automatically generated art. Increasingly more sophisticated generat…
Transframer: Arbitrary Frame Prediction with Generative Models
Charlie Nash, João Carreira, Jacob Walker +4
We present a general-purpose framework for image modelling and vision tasks based on probabilistic frame prediction. Our approach unifies a broad range of tasks, from image segment…
Gradient Forward-Propagation for Large-Scale Temporal Video Modelling
Mateusz Malinowski, Dimitrios Vytiniotis, Grzegorz Swirszcz +2
How can neural networks be trained on large-volume temporal data efficiently? To compute the gradients required to update parameters, backpropagation blocks computations until the…
The Visual QA Devil in the Details: The Impact of Early Fusion and Batch Norm on CLEVR
Mateusz Malinowski, Carl Doersch
Visual QA is a pivotal challenge for higher-level reasoning, requiring understanding language, vision, and relationships between many objects in a scene. Although datasets like CLE…
Answering Visual What-If Questions: From Actions to Predicted Scene Descriptions
M. Wagner, H. Basevi, R. Shetty +4
In-depth scene descriptions and question answering tasks have greatly increased the scope of today's definition of scene understanding. While such tasks are in principle open ended…
Learning Visual Question Answering by Bootstrapping Hard Attention
Mateusz Malinowski, Carl Doersch, Adam Santoro +1
Attention mechanisms in biological perception are thought to select subsets of perceptual information for more sophisticated processing which would be prohibitive to perform on all…
Long-Term Image Boundary Prediction
Apratim Bhattacharyya, Mateusz Malinowski, Bernt Schiele +1
Boundary estimation in images and videos has been a very active topic of research, and organizing visual information into boundaries and segments is believed to be a corner stone o…
Tutorial on Answering Questions about Images with Deep Learning
Mateusz Malinowski, Mario Fritz
Together with the development of more accurate methods in Computer Vision and Natural Language Understanding, holistic architectures that answer on questions about the content of r…
Mean Box Pooling: A Rich Image Representation and Output Embedding for the Visual Madlibs Task
Ashkan Mokarian, Mateusz Malinowski, Mario Fritz
We present Mean Box Pooling, a novel visual representation that pools over CNN representations of a large number, highly overlapping object proposals. We show that such representat…
Spatio-Temporal Image Boundary Extrapolation
Apratim Bhattacharyya, Mateusz Malinowski, Mario Fritz
Boundary prediction in images as well as video has been a very active topic of research and organizing visual information into boundaries and segments is believed to be a corner st…
Ask Your Neurons: A Deep Learning Approach to Visual Question Answering
Mateusz Malinowski, Marcus Rohrbach, Mario Fritz
We address a question answering task on real-world images that is set up as a Visual Turing Test. By combining latest advances in image representation and natural language processi…
Multi-Cue Zero-Shot Learning with Strong Supervision
Zeynep Akata, Mateusz Malinowski, Mario Fritz +1
Scaling up visual category recognition to large numbers of classes remains challenging. A promising research direction is zero-shot learning, which does not require any training da…
Ask Your Neurons: A Neural-based Approach to Answering Questions about Images
Mateusz Malinowski, Marcus Rohrbach, Mario Fritz
We address a question answering task on real-world images that is set up as a Visual Turing Test. By combining latest advances in image representation and natural language processi…
A Pooling Approach to Modelling Spatial Relations for Image Retrieval and Annotation
Mateusz Malinowski, Mario Fritz
Over the last two decades we have witnessed strong progress on modeling visual object classes, scenes and attributes that have significantly contributed to automated image understa…
Learnable Pooling Regions for Image Classification
Mateusz Malinowski, Mario Fritz
Biologically inspired, from the early HMAX model to Spatial Pyramid Matching, pooling has played an important role in visual recognition pipelines. Spatial pooling, by grouping of…