8 citations · 23 across the 16 of their papers we have counts for
19 papers
DepthWorld: 3D World Model for Robot Manipulation
Jai Bardhan, Josef Sivic, Vladimir Petrik
World models offer a data-driven alternative to traditional simulators for robotics, with applications spanning policy evaluation, improvement, and planning. All of these uses depe…
Memory-Guided B-Roll Generation from User Video Collections
Cusuh Ham, Fabian Caba Heilbron, Josef Sivic +1
We introduce an approach for collection-grounded B-roll sequence generation. Given a user's video collection, a directive given in natural language, and a target duration, the goal…
Autoregressive Modeling of Film with Applications in Video Montage
Marcelo Sandoval-Castañeda, Fabian Caba Heilbron, Shiry Ginosar +4
This work introduces FilmGPT, an autoregressive transformer designed to address the challenge of video montage--turning a collection of raw, "unwatchable" footage into coherent cin…
Temporally Consistent Object 6D Pose Estimation for Robot Control
Kateryna Zorina, Vojtech Priban, Mederic Fourmy +2
Single-view RGB object pose estimators have reached a level of precision and efficiency that makes them good candidates for vision-based robot control. However, off-the-shelf metho…
Persistent Robot World Models: Stabilizing Multi-Step Rollouts via Reinforcement Learning
Jai Bardhan, Patrik Drozdik, Josef Sivic +1
Action-conditioned robot world models generate future video frames of the manipulated scene given a robot action sequence, offering a promising alternative for simulating tasks tha…
REALM: A Real-to-Sim Validated Benchmark for Generalization in Robotic Manipulation
Martin Sedlacek, Pavlo Yefanov, Georgy Ponimatkin +7
Vision-Language-Action (VLA) models empower robots to understand and execute tasks described by natural language instructions. However, a key challenge lies in their ability to gen…
AlignPose: Generalizable 6D Pose Estimation via Multi-view Feature-metric Alignment
Anna Šárová Mikeštíková, Médéric Fourmy, Martin Cífka +2
Single-view RGB model-based object pose estimation methods achieve strong generalization but are fundamentally limited by depth ambiguity, clutter, and occlusions. Multi-view pose…
ResidualViT for Efficient Temporally Dense Video Encoding
Mattia Soldan, Fabian Caba Heilbron, Bernard Ghanem +2
Several video understanding tasks, such as natural language temporal video grounding, temporal activity localization, and audio description generation, require "temporally dense" r…
EditDuet: A Multi-Agent System for Video Non-Linear Editing
Marcelo Sandoval-Castaneda, Bryan Russell, Josef Sivic +2
Automated tools for video editing and assembly have applications ranging from filmmaking and advertisement to content creation for social media. Previous video editing work has mai…
Discovering Divergent Representations between Text-to-Image Models
Lisa Dunlap, Joseph E. Gonzalez, Trevor Darrell +3
In this paper, we investigate when and how visual representations learned by two different generative models diverge. Given two text-to-image models, our goal is to discover visual…
Improving Personalized Search with Regularized Low-Rank Parameter Updates
Fiona Ryan, Josef Sivic, Fabian Caba Heilbron +3
Personalized vision-language retrieval seeks to recognize new concepts (e.g. "my dog Fido") from only a few examples. This task is challenging because it requires not only learning…
Multi-step manipulation task and motion planning guided by video demonstration
Kateryna Zorina, David Kovar, Mederic Fourmy +5
This work aims to leverage instructional video to solve complex multi-step task-and-motion planning tasks in robotics. Towards this goal, we propose an extension of the well-establ…
PhysPose: Refining 6D Object Poses with Physical Constraints
Martin Malenický, Martin Cífka, Médéric Fourmy +4
Accurate 6D object pose estimation from images is a key problem in object-centric scene understanding, enabling applications in robotics, augmented reality, and scene reconstructio…
6D Object Pose Tracking in Internet Videos for Robotic Manipulation
Georgy Ponimatkin, Martin Cífka, Tomáš Souček +4
We seek to extract a temporally consistent 6D pose trajectory of a manipulated object from an Internet instructional video. This is a challenging set-up for current 6D pose estimat…
Generative Timelines for Instructed Visual Assembly
Alejandro Pardo, Jui-Hsien Wang, Bernard Ghanem +3
The objective of this work is to manipulate visual timelines (e.g. a video) through natural language instructions, making complex timeline editing tasks accessible to non-expert or…
POP-3D: Open-Vocabulary 3D Occupancy Prediction from Images
Antonin Vobecky, Oriane Siméoni, David Hurych +4
We describe an approach to predict open-vocabulary 3D semantic voxel occupancy map from input 2D images with the objective of enabling 3D grounding, segmentation and retrieval of f…
NewMove: Customizing text-to-video models with novel motions
Joanna Materzynska, Josef Sivic, Eli Shechtman +3
We introduce an approach for augmenting text-to-video generation models with customized motions, extending their capabilities beyond the motions depicted in the original training d…
FocalPose++: Focal Length and Object Pose Estimation via Render and Compare
Martin Cífka, Georgy Ponimatkin, Yann Labbé +4
We introduce FocalPose++, a neural render-and-compare method for jointly estimating the camera-object 6D pose and camera focal length given a single RGB input image depicting a kno…
Visually Guided Model Predictive Robot Control via 6D Object Pose Localization and Tracking
Mederic Fourmy, Vojtech Priban, Jan Kristof Behrens +3
The objective of this work is to enable manipulation tasks with respect to the 6D pose of a dynamically moving object using a camera mounted on a robot. Examples include maintainin…