3 citations · 7 across the 19 of their papers we have counts for
22 papers
ID-V2V: Identity-Preserving Video Restylization
Yuancheng Xu, Mingming He, Pablo Salamanca +5
In visual storytelling, human performances are central to creative intent and narrative meaning. However, preserving human identity and performance while enabling flexible visual e…
Go-with-the-Track: Video Compositing and Motion Control with Point Tracking
Koichi Namekata, Yash Kant, Zhizheng Liu +9
Filmmaking demands precise motion control and reference image compositing -- capabilities that existing methods treat separately. Point-track-conditioned image-to-video models rest…
Vista4D: Video Reshooting with 4D Point Clouds
Kuan Heng Lin, Zhizheng Liu, Pablo Salamanca +9
We present Vista4D, a robust and flexible video reshooting framework that grounds the input video and target cameras in a 4D point cloud. Specifically, given an input video, our me…
Vid2Avatar-Pro: Authentic Avatar from Videos in the Wild via Universal Prior
Chen Guo, Junxuan Li, Yash Kant +3
We present Vid2Avatar-Pro, a method to create photorealistic and animatable 3D human avatars from monocular in-the-wild videos. Building a high-quality avatar that supports animati…
Pippo: High-Resolution Multi-View Humans from a Single Image
Yash Kant, Ethan Weber, Jin Kyu Kim +6
We present Pippo, a generative model capable of producing 1K resolution dense turnaround videos of a person from a single casually clicked photo. Pippo is a multi-view diffusion tr…
Fillerbuster: Unified Generative Scene Completion Model for Casual Captures
Ethan Weber, Norman Müller, Yash Kant +4
We present Fillerbuster, a unified model that completes unknown regions of a 3D scene with a multi-view latent diffusion transformer. Casual captures are often sparse and miss surr…
SG-I2V: Self-Guided Trajectory Control in Image-to-Video Generation
Koichi Namekata, Sherwin Bahmani, Ziyi Wu +3
Methods for image-to-video generation have achieved impressive, photo-realistic quality. However, adjusting specific elements in generated videos, such as object motion or camera m…
Realistic Evaluation of Model Merging for Compositional Generalization
Derek Tam, Yash Kant, Brian Lester +2
Merging has become a widespread way to cheaply combine individual models into a single model that inherits their capabilities and attains better performance. This popularity has sp…
Pixel-Aligned Multi-View Generation with Depth Guided Decoder
Zhenggang Tang, Peiye Zhuang, Chaoyang Wang +5
The task of image-to-multi-view generation refers to generating novel views of an instance from a single image. Recent methods achieve this by extending text-to-image latent diffus…
SPAD : Spatially Aware Multiview Diffusers
Yash Kant, Ziyi Wu, Michael Vasilkovsky +7
We present SPAD, a novel approach for creating consistent multi-view images from text prompts or single images. To enable multi-view generation, we repurpose a pretrained 2D diffus…
AToM: Amortized Text-to-Mesh using 2D Diffusion
Guocheng Qian, Junli Cao, Aliaksandr Siarohin +12
We introduce Amortized Text-to-Mesh (AToM), a feed-forward text-to-mesh framework optimized across multiple text prompts simultaneously. In contrast to existing text-to-3D methods…
Virtual Pets: Animatable Animal Generation in 3D Scenes
Yen-Chi Cheng, Chieh Hubert Lin, Chaoyang Wang +5
Toward unlocking the potential of generative models in immersive 4D experiences, we introduce Virtual Pet, a novel pipeline to model realistic and diverse motions for target animal…
iNVS: Repurposing Diffusion Inpainters for Novel View Synthesis
Yash Kant, Aliaksandr Siarohin, Michael Vasilkovsky +4
We present a method for generating consistent novel views from a single source image. Our approach focuses on maximizing the reuse of visible pixels from the source image. To achie…
CAMM: Building Category-Agnostic and Animatable 3D Models from Monocular Videos
Tianshu Kuai, Akash Karthikeyan, Yash Kant +2
Animating an object in 3D often requires an articulated structure, e.g. a kinematic chain or skeleton of the manipulated object with proper skinning weights, to obtain smooth movem…
Invertible Neural Skinning
Yash Kant, Aliaksandr Siarohin, Riza Alp Guler +4
Building animatable and editable models of clothed humans from raw 3D scans and poses is a challenging problem. Existing reposing methods suffer from the limited expressiveness of…
Building Scalable Video Understanding Benchmarks through Sports
Aniket Agarwal, Alex Zhang, Karthik Narasimhan +3
Existing benchmarks for evaluating long video understanding falls short on two critical aspects, either lacking in scale or quality of annotations. These limitations arise from the…
LaTeRF: Label and Text Driven Object Radiance Fields
Ashkan Mirzaei, Yash Kant, Jonathan Kelly +1
Obtaining 3D object representations is important for creating photo-realistic simulations and for collecting AR and VR assets. Neural fields have shown their effectiveness in learn…
Housekeep: Tidying Virtual Households using Commonsense Reasoning
Yash Kant, Arun Ramachandran, Sriram Yenamandra +4
We introduce Housekeep, a benchmark to evaluate commonsense reasoning in the home for embodied AI. In Housekeep, an embodied agent must tidy a house by rearranging misplaced object…
NarrationBot and InfoBot: A Hybrid System for Automated Video Description
Shasta Ihorn, Yue-Ting Siu, Aditya Bodi +6
Video accessibility is crucial for blind and low vision users for equitable engagements in education, employment, and entertainment. Despite the availability of professional and am…
Contrast and Classify: Training Robust VQA Models
Yash Kant, Abhinav Moudgil, Dhruv Batra +2
Recent Visual Question Answering (VQA) models have shown impressive performance on the VQA benchmark but remain sensitive to small linguistic variations in input questions. Existin…
Spatially Aware Multimodal Transformers for TextVQA
Yash Kant, Dhruv Batra, Peter Anderson +4
Textual cues are essential for everyday tasks like buying groceries and using public transport. To develop this assistive technology, we study the TextVQA task, i.e., reasoning abo…
ICLR Reproducibility Challenge Report (Padam : Closing The Generalization Gap Of Adaptive Gradient Methods in Training Deep Neural Networks)
Harshal Mittal, Kartikey Pandey, Yash Kant
This work is a part of ICLR Reproducibility Challenge 2019, we try to reproduce the results in the conference submission PADAM: Closing The Generalization Gap of Adaptive Gradient…