57 citations · 111 across the 20 of their papers we have counts for
22 papers
Rays as Pixels: Learning A Joint Distribution of Videos and Camera Trajectories
Wonbong Jang, Shikun Liu, Soubhik Sanyal +6
Recovering camera parameters from images and rendering scenes from novel viewpoints have been treated as separate tasks in computer vision and graphics. This separation breaks down…
HiStream: Efficient High-Resolution Video Generation via Redundancy-Eliminated Streaming
Haonan Qiu, Shikun Liu, Zijian Zhou +10
High-resolution video generation, while crucial for digital media and film, is computationally bottlenecked by the quadratic complexity of diffusion models, making practical infere…
TUNA: Taming Unified Visual Representations for Native Unified Multimodal Models
Zhiheng Liu, Weiming Ren, Haozhe Liu +22
Unified multimodal models (UMMs) aim to jointly perform multimodal understanding and generation within a single framework. We present TUNA, a native UMM that builds a unified conti…
Mixture of States: Routing Token-Level Dynamics for Multimodal Generation
Haozhe Liu, Ding Liu, Mingchen Zhuge +16
We introduce MoS (Mixture of States), a novel fusion paradigm for multimodal diffusion models that merges modalities using flexible, state-based interactions. The core of MoS is a…
Hyper-VolTran: Fast and Generalizable One-Shot Image to 3D Object Structure via HyperNetworks
Christian Simon, Sen He, Juan-Manuel Perez-Rua +3
Solving image-to-3D from a single view is an ill-posed problem, and current neural reconstruction methods addressing it through diffusion models still rely on scene-specific optimi…
GenTron: Diffusion Transformers for Image and Video Generation
Shoufa Chen, Mengmeng Xu, Jiawei Ren +7
In this study, we explore Transformer-based diffusion models for image and video generation. Despite the dominance of Transformer architectures in various fields due to their flexi…
FLATTEN: optical FLow-guided ATTENtion for consistent text-to-video editing
Yuren Cong, Mengmeng Xu, Christian Simon +7
Text-to-video editing aims to edit the visual appearance of a source video conditional on textual prompts. A major challenge in this task is to ensure that all frames in the edited…
Multi-Modal Few-Shot Temporal Action Detection
Sauradip Nag, Mengmeng Xu, Xiatian Zhu +4
Few-shot (FS) and zero-shot (ZS) learning are two different approaches for scaling temporal action detection (TAD) to new classes. The former adapts a pretrained vision model to a…
Where is my Wallet? Modeling Object Proposal Sets for Egocentric Visual Query Localization
Mengmeng Xu, Yanghao Li, Cheng-Yang Fu +3
This paper deals with the problem of localizing objects in image and video datasets from visual exemplars. In particular, we focus on the challenging problem of egocentric visual q…
Negative Frames Matter in Egocentric Visual Query 2D Localization
Mengmeng Xu, Cheng-Yang Fu, Yanghao Li +3
The recently released Ego4D dataset and benchmark significantly scales and diversifies the first-person visual perception data. In Ego4D, the Visual Queries 2D Localization task ai…
Sylph: A Hypernetwork Framework for Incremental Few-shot Object Detection
Li Yin, Juan M Perez-Rua, Kevin J Liang
We study the challenging incremental few-shot object detection (iFSD) setting. Recently, hypernetwork-based approaches have been studied in the context of continuous and finetune-f…
SAIC_Cambridge-HuPBA-FBK Submission to the EPIC-Kitchens-100 Action Recognition Challenge 2021
Swathikiran Sudhakaran, Adrian Bulat, Juan-Manuel Perez-Rua +5
This report presents the technical details of our submission to the EPIC-Kitchens-100 Action Recognition Challenge 2021. To participate in the challenge we deployed spatio-temporal…
Space-time Mixing Attention for Video Transformer
Adrian Bulat, Juan-Manuel Perez-Rua, Swathikiran Sudhakaran +2
This paper is on video recognition using Transformers. Very recent attempts in this area have demonstrated promising results in terms of recognition accuracy, yet they have been al…
TNT: Text-Conditioned Network with Transductive Inference for Few-Shot Video Classification
Andrés Villa, Juan-Manuel Perez-Rua, Victor Escorcia +3
Recently, few-shot video classification has received an increasing interest. Current approaches mostly focus on effectively exploiting the temporal dimension in videos to improve l…
Boundary-sensitive Pre-training for Temporal Localization in Videos
Mengmeng Xu, Juan-Manuel Perez-Rua, Victor Escorcia +5
Many video analysis tasks require temporal localization thus detection of content changes. However, most existing models developed for these tasks are pre-trained on general video…
Egocentric Action Recognition by Video Attention and Temporal Context
Juan-Manuel Perez-Rua, Antoine Toisoul, Brais Martinez +4
We present the submission of Samsung AI Centre Cambridge to the CVPR2020 EPIC-Kitchens Action Recognition Challenge. In this challenge, action recognition is posed as the problem o…
Knowing What, Where and When to Look: Efficient Video Action Modeling with Attention
Juan-Manuel Perez-Rua, Brais Martinez, Xiatian Zhu +3
Attentive video modeling is essential for action recognition in unconstrained videos due to their rich yet redundant information over space and time. However, introducing attention…
Incremental Few-Shot Object Detection
Juan-Manuel Perez-Rua, Xiatian Zhu, Timothy Hospedales +1
Most existing object detection methods rely on the availability of abundant labelled training samples per class and offline model training in a batch mode. These requirements subst…
MFAS: Multimodal Fusion Architecture Search
Juan-Manuel Pérez-Rúa, Valentin Vielzeuf, Stéphane Pateux +2
We tackle the problem of finding good architectures for multimodal classification problems. We propose a novel and generic search space that spans a large number of possible fusion…
Efficient Progressive Neural Architecture Search
Juan-Manuel Perez-Rua, Moez Baccouche, Stephane Pateux
This paper addresses the difficult problem of finding an optimal neural architecture design for a given image classification task. We propose a method that aggregates two main resu…
Learning how to be robust: Deep polynomial regression
Juan-Manuel Perez-Rua, Tomas Crivelli, Patrick Bouthemy +1
Polynomial regression is a recurrent problem with a large number of applications. In computer vision it often appears in motion analysis. Whatever the application, standard methods…
ROAM: a Rich Object Appearance Model with Application to Rotoscoping
Ondrej Miksik, Juan-Manuel Pérez-Rúa, Philip H. S. Torr +1
Rotoscoping, the detailed delineation of scene elements through a video shot, is a painstaking task of tremendous importance in professional post-production pipelines. While pixel-…