49 citations · 163 across the 22 of their papers we have counts for
23 papers · 1 filter
Token-Shuffle: Towards High-Resolution Image Generation with Autoregressive Models
Xu Ma, Peize Sun, Haoyu Ma +22
Autoregressive (AR) models, long dominant in language generation, are increasingly applied to image synthesis but are often considered less competitive than Diffusion-based models.…
Autoregressive Distillation of Diffusion Transformers
Yeongmin Kim, Sotiris Anagnostidis, Yuming Du +6
Diffusion models with transformer architectures have demonstrated promising capabilities in generating high-fidelity images and scalability for high resolution. However, iterative…
XR-MBT: Multi-modal Full Body Tracking for XR through Self-Supervision with Learned Depth Point Cloud Registration
Denys Rozumnyi, Nadine Bertsch, Othman Sbai +6
Tracking the full body motions of users in XR (AR/VR) devices is a fundamental challenge to bring a sense of authentic social presence. Due to the absence of dedicated leg sensors,…
Movie Gen: A Cast of Media Foundation Models
Adam Polyak, Amit Zohar, Andrew Brown +85
We present Movie Gen, a cast of foundation models that generates high-quality, 1080p HD videos with different aspect ratios and synchronized audio. We also show additional capabili…
Imagine Flash: Accelerating Emu Diffusion Models with Backward Distillation
Jonas Kohler, Albert Pumarola, Edgar Schönfeld +4
Diffusion models are a powerful generative framework, but come with expensive inference. Existing acceleration methods often compromise image quality or fail under complex conditio…
Using Motion Cues to Supervise Single-Frame Body Pose and Shape Estimation in Low Data Regimes
Andrey Davydov, Alexey Sidnev, Artsiom Sanakoyeu +3
When enough annotated training data is available, supervised deep-learning algorithms excel at estimating human body pose and shape using a single camera. The effects of too little…
Cache Me if You Can: Accelerating Diffusion Models through Block Caching
Felix Wimbauer, Bichen Wu, Edgar Schoenfeld +11
Diffusion models have recently revolutionized the field of image synthesis due to their ability to generate photorealistic images. However, one of the major drawbacks of diffusion…
BoDiffusion: Diffusing Sparse Observations for Full-Body Human Motion Synthesis
Angela Castillo, Maria Escobar, Guillaume Jeanneret +4
Mixed reality applications require tracking the user's full-body motion to enable an immersive experience. However, typical head-mounted devices can only track head and hand moveme…
Avatars Grow Legs: Generating Smooth Human Motion from Sparse Tracking Inputs with Diffusion Model
Yuming Du, Robin Kips, Albert Pumarola +3
With the recent surge in popularity of AR/VR applications, realistic and accurate control of 3D full-body avatars has become a highly demanded feature. A particular challenge is th…
VisCo Grids: Surface Reconstruction with Viscosity and Coarea Grids
Albert Pumarola, Artsiom Sanakoyeu, Lior Yariv +2
Surface reconstruction has been seeing a lot of progress lately by utilizing Implicit Neural Representations (INRs). Despite their success, INRs often introduce hard to control ind…
Re-ReND: Real-time Rendering of NeRFs across Devices
Sara Rojas, Jesus Zarzar, Juan Camilo Perez +4
This paper proposes a novel approach for rendering a pre-trained Neural Radiance Field (NeRF) in real-time on resource-constrained devices. We introduce Re-ReND, a method enabling…
MotionCNN: A Strong Baseline for Motion Prediction in Autonomous Driving
Stepan Konev, Kirill Brodt, Artsiom Sanakoyeu
To plan a safe and efficient route, an autonomous vehicle should anticipate future motions of other agents around it. Motion prediction is an extremely challenging task that recent…
Improving Deep Metric Learning by Divide and Conquer
Artsiom Sanakoyeu, Pingchuan Ma, Vadim Tschernezki +1
Deep metric learning (DML) is a cornerstone of many computer vision applications. It aims at learning a mapping from the input domain to an embedding space, where semantically simi…
Discovering Relationships between Object Categories via Universal Canonical Maps
Natalia Neverova, Artsiom Sanakoyeu, Patrick Labatut +2
We tackle the problem of learning the geometry of multiple categories of deformable objects jointly. Recent work has shown that it is possible to learn a unified dense pose predict…
Self-supervised Depth Denoising Using Lower- and Higher-quality RGB-D sensors
Akhmedkhan Shabanov, Ilya Krotov, Nikolay Chinaev +7
Consumer-level depth cameras and depth sensors embedded in mobile devices enable numerous applications, such as AR games and face identification. However, the quality of the captur…
A Content Transformation Block For Image Style Transfer
Dmytro Kotovenko, Artsiom Sanakoyeu, Pingchuan Ma +2
Style transfer has recently received a lot of attention, since it allows to study fundamental challenges in image understanding and synthesis. Recent work has significantly improve…
Transferring Dense Pose to Proximal Animal Classes
Artsiom Sanakoyeu, Vasil Khalidov, Maureen S. McCarthy +2
Recent contributions have demonstrated that it is possible to recognize the pose of humans densely and accurately given a large dataset of poses annotated in detail. In principle,…
Divide and Conquer the Embedding Space for Metric Learning
Artsiom Sanakoyeu, Vadim Tschernezki, Uta Büchler +1
Learning the embedding space, where semantically similar objects are located close together and dissimilar objects far apart, is a cornerstone of many computer vision applications.…
Semi-Supervised Segmentation of Salt Bodies in Seismic Images using an Ensemble of Convolutional Neural Networks
Yauhen Babakhin, Artsiom Sanakoyeu, Hirotoshi Kitamura
Seismic image analysis plays a crucial role in a wide range of industrial applications and has been receiving significant attention. One of the essential challenges of seismic imag…
A Style-Aware Content Loss for Real-time HD Style Transfer
Artsiom Sanakoyeu, Dmytro Kotovenko, Sabine Lang +1
Recently, style transfer has received a lot of attention. While much of this research has aimed at speeding up processing, the approaches are still lacking from a principled, art h…
Deep Unsupervised Learning of Visual Similarities
Artsiom Sanakoyeu, Miguel A. Bautista, Björn Ommer
Exemplar learning of visual similarities in an unsupervised manner is a problem of paramount importance to Computer Vision. In this context, however, the recent breakthrough in dee…
Deep Unsupervised Similarity Learning using Partially Ordered Sets
Miguel A Bautista, Artsiom Sanakoyeu, Björn Ommer
Unsupervised learning of visual similarities is of paramount importance to computer vision, particularly due to lacking training data for fine-grained similarities. Deep learning o…
CliqueCNN: Deep Unsupervised Exemplar Learning
Miguel A. Bautista, Artsiom Sanakoyeu, Ekaterina Sutter +1
Exemplar learning is a powerful paradigm for discovering visual similarities in an unsupervised manner. In this context, however, the recent breakthrough in deep learning could not…