Publications (23)
Multi-Track Timeline Control for Text-Driven 3D Human Motion Generation
Mathis Petrovich, Or Litany, Umar Iqbal +4
Recent advances in generative modeling have led to promising progress on synthesizing 3D human motion from text, with methods that can generate character animations from short prom…
Multiview Aggregation for Learning Category-Specific Shape Reconstruction
Srinath Sridhar, Davis Rempe, Julien Valentin +2
We investigate the problem of learning category-specific 3D shape reconstruction from a variable number of RGB views of previously unobserved object instances. Most approaches for…
NIFTY: Neural Object Interaction Fields for Guided Human Motion Synthesis
Nilesh Kulkarni, Davis Rempe, Kyle Genova +4
We address the problem of generating realistic 3D motions of humans interacting with objects in a scene. Our key idea is to create a neural interaction field attached to a specific…
MotionBricks: Scalable Real-Time Motions with Modular Latent Generative Model and Smart Primitives
Tingwu Wang, Olivier Dionne, Michael De Ruyter +13
Despite transformative advances in generative motion synthesis, real-time interactive motion control remains dominated by traditional techniques. In this work, we identify two key…
Contact and Human Dynamics from Monocular Video
Davis Rempe, Leonidas J. Guibas, Aaron Hertzmann +3
Existing deep models predict 2D and 3D kinematic poses from video that are approximately accurate, but contain visible errors that violate physical constraints, such as feet penetr…
ARDY: Autoregressive Diffusion with Hybrid Representation for Interactive Human Motion Generation
Kaifeng Zhao, Mathis Petrovich, Haotian Zhang +3
Generating realistic 3D human motions in real-time within interactive applications is key for animation, simulation, and humanoid robotics. While recent offline motion generation a…
Generating Useful Accident-Prone Driving Scenarios via a Learned Traffic Prior
Davis Rempe, Jonah Philion, Leonidas J. Guibas +2
Evaluating and improving planning for autonomous vehicles requires scalable generation of long-tail traffic scenarios. To be useful, these scenarios must be realistic and challengi…
GENMO: A GENeralist Model for Human MOtion
Jiefeng Li, Jinkun Cao, Haotian Zhang +4
Human motion modeling traditionally separates motion generation and estimation into distinct tasks with specialized models. Motion generation models focus on creating diverse, real…
CurveCloudNet: Processing Point Clouds with 1D Structure
Colton Stearns, Davis Rempe, Jiateng Liu +5
Modern depth sensors such as LiDAR operate by sweeping laser-beams across the scene, resulting in a point cloud with notable 1D curve-like structures. In this work, we introduce a…
HuMoR: 3D Human Motion Model for Robust Pose Estimation
Davis Rempe, Tolga Birdal, Aaron Hertzmann +3
We introduce HuMoR: a 3D Human Motion Model for Robust Estimation of temporal pose and shape. Though substantial progress has been made in estimating 3D human motion and shape from…
COPILOT: Human-Environment Collision Prediction and Localization from Egocentric Videos
Boxiao Pan, Bokui Shen, Davis Rempe +4
The ability to forecast human-environment collisions from egocentric observations is vital to enable collision avoidance in applications such as VR, AR, and wearable assistive robo…
HIL: Hybrid Imitation Learning of Diverse Parkour Skills from Videos
Jiashun Wang, Yifeng Jiang, Haotian Zhang +4
Data-driven methods leveraging deep reinforcement learning have become the dominant paradigm for developing controllers that enable physically simulated characters to produce natur…
SpOT: Spatiotemporal Modeling for 3D Object Tracking
Colton Stearns, Davis Rempe, Jie Li +5
3D multi-object tracking aims to uniquely and consistently identify all mobile entities through time. Despite the rich spatiotemporal information available in this setting, current…
Learning Generalizable Physical Dynamics of 3D Rigid Objects
Davis Rempe, Srinath Sridhar, He Wang +1
Humans have a remarkable ability to predict the effect of physical interactions on the dynamics of objects. Endowing machines with this ability would allow important applications i…
Language-Guided Traffic Simulation via Scene-Level Diffusion
Ziyuan Zhong, Davis Rempe, Yuxiao Chen +5
Realistic and controllable traffic simulation is a core capability that is necessary to accelerate autonomous vehicle (AV) development. However, current approaches for controlling…
Trace and Pace: Controllable Pedestrian Animation via Guided Trajectory Diffusion
Davis Rempe, Zhengyi Luo, Xue Bin Peng +5
We introduce a method for generating realistic pedestrian trajectories and full-body animations that can be controlled to meet user-defined goals. We draw on recent advances in gui…
Generating Human Interaction Motions in Scenes with Text Control
Hongwei Yi, Justus Thies, Michael J. Black +2
We present TeSMo, a method for text-controlled scene-aware motion generation based on denoising diffusion models. Previous text-to-motion methods focus on characters in isolation w…
A Point-Cloud Deep Learning Framework for Prediction of Fluid Flow Fields on Irregular Geometries
Ali Kashefi, Davis Rempe, Leonidas J. Guibas
We present a novel deep learning framework for flow field predictions in irregular domains when the solution is a function of the geometry of either the domain or objects inside th…
Kimodo: Scaling Controllable Human Motion Generation
Davis Rempe, Mathis Petrovich, Ye Yuan +21
High-quality human motion data is becoming increasingly important for applications in robotics, simulation, and entertainment. Recent generative models offer a potential data sourc…
Guided Conditional Diffusion for Controllable Traffic Simulation
Ziyuan Zhong, Davis Rempe, Danfei Xu +5
Controllable and realistic traffic simulation is critical for developing and verifying autonomous vehicles. Typical heuristic-based traffic models offer flexible control to make ve…
CaSPR: Learning Canonical Spatiotemporal Point Cloud Representations
Davis Rempe, Tolga Birdal, Yongheng Zhao +3
We propose CaSPR, a method to learn object-centric Canonical Spatiotemporal Point Cloud Representations of dynamically moving or evolving objects. Our goal is to enable information…
COIN: Control-Inpainting Diffusion Prior for Human and Camera Motion Estimation
Jiefeng Li, Ye Yuan, Davis Rempe +5
Estimating global human motion from moving cameras is challenging due to the entanglement of human and camera motions. To mitigate the ambiguity, existing methods leverage learned…
Predicting the Physical Dynamics of Unseen 3D Objects
Davis Rempe, Srinath Sridhar, He Wang +1
Machines that can predict the effect of physical interactions on the dynamics of previously unseen object instances are important for creating better robots and interactive virtual…