7 citations · 18 across the 20 of their papers we have counts for
24 papers
SCOPD: Sparse-Context On-Policy Self-Distillation for Efficient Vision-Language Models
Ahmadreza Jeddi, Enming Zhang, Jasper Gerigk +12
Reasoning vision-language models (VLMs) process images and videos as long sequences of visual tokens, making inference expensive. Training-free token pruning reduces this cost, but…
Towards Unified Dynamic Face Landmark Detection
Sebastian Regalado, Varshanth R. Rao, Ruowei Jiang +2
Although advancements in face landmark detection (FLD) methods continue to push performance boundaries, they overlook two major functional limitations: (1) different network parame…
Squeezing Capacity from Multimodal Large Language Models for Subject-driven Generation
Shuhong Zheng, Aashish Kumar Misraa, Yu-Teng Li +2
Subject-driven image generation aims to synthesize new images that preserve the identity of the given subject while following textual instructions. Existing approaches often encode…
Good Token Hunting: A Hitchhiker's Guide to Token Selection for Visual Geometry Transformers
Shuhong Zheng, Michael Oechsle, Erik Sandström +3
Visual geometry transformers have become powerful architectures for multi-view 3D reconstruction, enabling joint prediction of multiple 3D attributes in a feed-forward manner. Howe…
ActionParty: Multi-Subject Action Binding in Generative Video Games
Alexander Pondaven, Ziyi Wu, Igor Gilitschenski +4
Recent advances in video diffusion have enabled the development of "world models" capable of simulating interactive environments. However, these models are largely restricted to si…
Material Magic Wand: Material-Aware Grouping of 3D Parts in Untextured Meshes
Umangi Jain, Vladimir Kim, Matheus Gadelha +2
We introduce the problem of material-aware part grouping in untextured meshes. Many real-world shapes, such as scales of pinecones or windows of buildings, contain repeated structu…
TESPEC: Temporally-Enhanced Self-Supervised Pretraining for Event Cameras
Mohammad Mohammadi, Ziyi Wu, Igor Gilitschenski
Long-term temporal information is crucial for event-based perception tasks, as raw events only encode pixel brightness changes. Recent works show that when trained from scratch, re…
SEdit: Text-Guided Image Editing with Precise Semantic and Spatial Control
Xudong Liu, Zikun Chen, Ruowei Jiang +5
Recent advances in diffusion models have enabled high-quality generation and manipulation of images guided by texts, as well as concept learning from images. However, naive applica…
DenseDPO: Fine-Grained Temporal Preference Optimization for Video Diffusion Models
Ziyi Wu, Anil Kag, Ivan Skorokhodov +5
Direct Preference Optimization (DPO) has recently been applied as a post-training technique for text-to-video diffusion models. To obtain training data, annotators are asked to pro…
Pippo: High-Resolution Multi-View Humans from a Single Image
Yash Kant, Ethan Weber, Jin Kyu Kim +6
We present Pippo, a generative model capable of producing 1K resolution dense turnaround videos of a person from a single casually clicked photo. Pippo is a multi-view diffusion tr…
Mind the Time: Temporally-Controlled Multi-Event Video Generation
Ziyi Wu, Aliaksandr Siarohin, Willi Menapace +5
Real-world videos consist of sequences of events. Generating such sequences with precise temporal control is infeasible with existing video generators that rely on a single paragra…
SG-I2V: Self-Guided Trajectory Control in Image-to-Video Generation
Koichi Namekata, Sherwin Bahmani, Ziyi Wu +3
Methods for image-to-video generation have achieved impressive, photo-realistic quality. However, adjusting specific elements in generated videos, such as object motion or camera m…
Realistic Evaluation of Model Merging for Compositional Generalization
Derek Tam, Yash Kant, Brian Lester +2
Merging has become a widespread way to cheaply combine individual models into a single model that inherits their capabilities and attains better performance. This popularity has sp…
Neural Assets: 3D-Aware Multi-Object Scene Synthesis with Image Diffusion Models
Ziyi Wu, Yulia Rubanova, Rishabh Kabra +6
We address the problem of multi-object 3D pose control in image diffusion models. Instead of conditioning on a sequence of text tokens, we propose to use a set of per-object repres…
SPAD : Spatially Aware Multiview Diffusers
Yash Kant, Ziyi Wu, Michael Vasilkovsky +7
We present SPAD, a novel approach for creating consistent multi-view images from text prompts or single images. To enable multi-view generation, we repurpose a pretrained 2D diffus…
AToM: Amortized Text-to-Mesh using 2D Diffusion
Guocheng Qian, Junli Cao, Aliaksandr Siarohin +12
We introduce Amortized Text-to-Mesh (AToM), a feed-forward text-to-mesh framework optimized across multiple text prompts simultaneously. In contrast to existing text-to-3D methods…
LEOD: Label-Efficient Object Detection for Event Cameras
Ziyi Wu, Mathias Gehrig, Qing Lyu +2
Object detection with event cameras benefits from the sensor's low latency and high dynamic range. However, it is costly to fully label event streams for supervised training due to…
iNVS: Repurposing Diffusion Inpainters for Novel View Synthesis
Yash Kant, Aliaksandr Siarohin, Michael Vasilkovsky +4
We present a method for generating consistent novel views from a single source image. Our approach focuses on maximizing the reuse of visible pixels from the source image. To achie…
EventCLIP: Adapting CLIP for Event-based Object Recognition
Ziyi Wu, Xudong Liu, Igor Gilitschenski
Recent advances in zero-shot and few-shot classification heavily rely on the success of pre-trained vision-language models (VLMs) such as CLIP. Due to a shortage of large-scale dat…
SlotDiffusion: Object-Centric Generative Modeling with Diffusion Models
Ziyi Wu, Jingyu Hu, Wuyue Lu +2
Object-centric learning aims to represent visual data with a set of object entities (a.k.a. slots), providing structured representations that enable systematic generalization. Leve…
CAMM: Building Category-Agnostic and Animatable 3D Models from Monocular Videos
Tianshu Kuai, Akash Karthikeyan, Yash Kant +2
Animating an object in 3D often requires an articulated structure, e.g. a kinematic chain or skeleton of the manipulated object with proper skinning weights, to obtain smooth movem…
Invertible Neural Skinning
Yash Kant, Aliaksandr Siarohin, Riza Alp Guler +4
Building animatable and editable models of clothed humans from raw 3D scans and poses is a challenging problem. Existing reposing methods suffer from the limited expressiveness of…
Building Scalable Video Understanding Benchmarks through Sports
Aniket Agarwal, Alex Zhang, Karthik Narasimhan +3
Existing benchmarks for evaluating long video understanding falls short on two critical aspects, either lacking in scale or quality of annotations. These limitations arise from the…
Housekeep: Tidying Virtual Households using Commonsense Reasoning
Yash Kant, Arun Ramachandran, Sriram Yenamandra +4
We introduce Housekeep, a benchmark to evaluate commonsense reasoning in the home for embodied AI. In Housekeep, an embodied agent must tidy a house by rearranging misplaced object…