Publications (108)
Emergent Active Perception and Dexterity of Simulated Humanoids from Visual Reinforcement Learning
Zhengyi Luo, Chen Tessler, Toru Lin +8
InterPet4D: A Multimodal 4D Human-Pet Interaction Dataset for Pet Motion Generation
Yichen Peng, Jyun-Ting Song, Chen-Chieh Liao +3
Monocular 3D Object Detection with Pseudo-LiDAR Point Cloud
Xinshuo Weng, Kris Kitani
G-HOP: Generative Hand-Object Prior for Interaction Reconstruction and Grasp Synthesis
Yufei Ye, Abhinav Gupta, Kris Kitani +1
Evaluating a VR System for Collecting Safety-Critical Vehicle-Pedestrian Interactions
Erica Weng, Kenta Mukoya, Deva Ramanan +1
ATLAS: Decoupling Skeletal and Shape Parameters for Expressive Parametric Human Modeling
Jinhyung Park, Javier Romero, Shunsuke Saito +7
Human Action Anticipation: A Survey
Bolin Lai, Sam Toyer, Tushar Nagarajan +7
Ego4D: Around the World in 3,000 Hours of Egocentric Video
Kristen Grauman, Andrew Westbury, Eugene Byrne +82
Zero-Shot Multi-Object Scene Completion
Shun Iwase, Katherine Liu, Vitor Guizilini +4
JaywalkerVR: A VR System for Collecting Safety-Critical Pedestrian-Vehicle Interactions
Kenta Mukoya, Erica Weng, Rohan Choudhury +1
Multi-Object Tracking by Hierarchical Visual Representations
Jinkun Cao, Jiangmiao Pang, Kris Kitani
DanceTrack: Multi-Object Tracking in Uniform Appearance and Diverse Motion
Peize Sun, Jinkun Cao, Yi Jiang +4
Improving Lesion Segmentation for Diabetic Retinopathy using Adversarial Learning
Qiqi Xiao, Jiaxu Zou, Muqiao Yang +5
ASAP: Aligning Simulation and Real-World Physics for Learning Agile Humanoid Whole-Body Skills
Tairan He, Jiawei Gao, Wenli Xiao +15
Generalizable Neural Human Renderer
Mana Masuda, Jinhyung Park, Shun Iwase +2
TEMPO: Efficient Multi-View Pose Estimation, Tracking, and Forecasting
Rohan Choudhury, Kris Kitani, Laszlo A. Jeni
Understanding hand-object manipulation by modeling the contextual relationship between actions, grasp types and object attributes
Minjie Cai, Kris Kitani, Yoichi Sato
Perpetual Humanoid Control for Real-time Simulated Avatars
Zhengyi Luo, Jinkun Cao, Alexander Winkler +2
Estimating 3D Camera Pose from 2D Pedestrian Trajectories
Yan Xu, Vivek Roy, Kris Kitani
AutoSelect: Automatic and Dynamic Detection Selection for 3D Multi-Object Tracking
Xinshuo Weng, Kris Kitani
Track Targets by Dense Spatio-Temporal Position Encoding
Jinkun Cao, Hao Wu, Kris Kitani
DyaDiT: A Multi-Modal Diffusion Transformer for Socially Favorable Dyadic Gesture Generation
Yichen Peng, Jyun-Ting Song, Siyeol Jung +7
MHR: Momentum Human Rig
Aaron Ferguson, Ahmed A. A. Osman, Berta Bescos +44
Efficient Non-Line-of-Sight Imaging from Transient Sinograms
Mariko Isogawa, Dorian Chan, Ye Yuan +2
3D-Aware Encoding for Style-based Neural Radiance Fields
Yu-Jhe Li, Tao Xu, Bichen Wu +6
Embodied Scene-aware Human Pose Estimation
Zhengyi Luo, Shun Iwase, Ye Yuan +1
PTP: Parallelized Tracking and Prediction with Graph Neural Networks and Diversity Sampling
Xinshuo Weng, Ye Yuan, Kris Kitani
Visual Compiler: Synthesizing a Scene-Specific Pedestrian Detector and Pose Estimator
Namhoon Lee, Xinshuo Weng, Vishnu Naresh Boddeti +4
REST3D: Reconstructing Physically Stable 3D Scenes from a Single Image
Xiaoxuan Ma, Jiashun Wang, Nicolas Ugrinovic +2
Residual Force Control for Agile Human Behavior Imitation and Extended Motion Synthesis
Ye Yuan, Kris Kitani
Learning Spatio-Temporal Features with Two-Stream Deep 3D CNNs for Lipreading
Xinshuo Weng, Kris Kitani
Trace and Pace: Controllable Pedestrian Animation via Guided Trajectory Diffusion
Davis Rempe, Zhengyi Luo, Xue Bin Peng +5
EgoHumans: An Egocentric 3D Multi-Human Benchmark
Rawal Khirodkar, Aayush Bansal, Lingni Ma +3
Cross-Domain Adaptive Teacher for Object Detection
Yu-Jhe Li, Xiaoliang Dai, Chih-Yao Ma +6
Joint Object Detection and Multi-Object Tracking with Graph Neural Networks
Yongxin Wang, Kris Kitani, Xinshuo Weng
AEI: Actors-Environment Interaction with Adaptive Attention for Temporal Action Proposals Generation
Khoa Vo, Hyekang Joo, Kashu Yamazaki +4
GLAMR: Global Occlusion-Aware Human Mesh Recovery with Dynamic Cameras
Ye Yuan, Umar Iqbal, Pavlo Molchanov +2
Inverting the Pose Forecasting Pipeline with SPF2: Sequential Pointcloud Forecasting for Sequential Pose Forecasting
Xinshuo Weng, Jianren Wang, Sergey Levine +2
Audio-Visual Self-Supervised Terrain Type Discovery for Mobile Platforms
Akiyoshi Kurobe, Yoshikatsu Nakajima, Hideo Saito +1
End-to-End 3D Multi-Object Tracking and Trajectory Forecasting
Xinshuo Weng, Ye Yuan, Kris Kitani
DLow: Diversifying Latent Flows for Diverse Human Motion Prediction
Ye Yuan, Kris Kitani
Wide-Baseline Multi-Camera Calibration using Person Re-Identification
Yan Xu, Yu-Jhe Li, Xinshuo Weng +1
Efficient Model Performance Estimation via Feature Histories
Shengcao Cao, Xiaofang Wang, Kris Kitani
Optical Non-Line-of-Sight Physics-based 3D Human Pose Estimation
Mariko Isogawa, Ye Yuan, Matthew O'Toole +1
Graph Neural Networks for 3D Multi-Object Tracking
Xinshuo Weng, Yongxin Wang, Yunze Man +1
Multi-View Person Matching and 3D Pose Estimation with Arbitrary Uncalibrated Camera Networks
Yan Xu, Kris Kitani
Occluded Human Mesh Recovery
Rawal Khirodkar, Shashank Tripathi, Kris Kitani
SimPoE: Simulated Character Control for 3D Human Pose Estimation
Ye Yuan, Shih-En Wei, Tomas Simon +2
Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectives
Kristen Grauman, Andrew Westbury, Lorenzo Torresani +98
Deep OC-SORT: Multi-Pedestrian Tracking by Adaptive Re-Identification
Gerard Maggiolino, Adnan Ahmad, Jinkun Cao +1
Multi-Echo LiDAR for 3D Object Detection
Yunze Man, Xinshuo Weng, Prasanna Kumar Sivakuma +2
AB3DMOT: A Baseline for 3D Multi-Object Tracking and New Evaluation Metrics
Xinshuo Weng, Jianren Wang, David Held +1
Observation-Centric SORT: Rethinking SORT for Robust Multi-Object Tracking
Jinkun Cao, Jiangmiao Pang, Xinshuo Weng +2
Inverse Reinforcement Learning with Explicit Policy Estimates
Navyata Sanghvi, Shinnosuke Usami, Mohit Sharma +2
Personalized Dynamics Models for Adaptive Assistive Navigation Systems
Eshed Ohn-Bar, Kris Kitani, Chieko Asakawa
DegustaBot: Zero-Shot Visual Preference Estimation for Personalized Multi-Object Rearrangement
Benjamin A. Newman, Pranay Gupta, Kris Kitani +3
Rotational Rectification Network: Enabling Pedestrian Detection for Mobile Vision
Xinshuo Weng, Shangxuan Wu, Fares Beainy +1
Type-to-Track: Retrieve Any Object via Prompt-based Tracking
Pha Nguyen, Kha Gia Quach, Kris Kitani +1
No-Reference Image Quality Assessment via Feature Fusion and Multi-Task Learning
S. Alireza Golestaneh, Kris Kitani
Multi-Person 3D Pose Estimation from Multi-View Uncalibrated Depth Cameras
Yu-Jhe Li, Yan Xu, Rawal Khirodkar +2
Real-Time Simulated Avatar from Head-Mounted Sensors
Zhengyi Luo, Jinkun Cao, Rawal Khirodkar +4
3D-CLFusion: Fast Text-to-3D Rendering with Contrastive Latent Diffusion
Yu-Jhe Li, Tao Xu, Ji Hou +6
GroundNet: Monocular Ground Plane Normal Estimation with Geometric Consistency
Yunze Man, Xinshuo Weng, Xi Li +1
Joint Diffusion for Universal Hand-Object Grasp Generation
Jinkun Cao, Jingyuan Liu, Kris Kitani +1
IDOL: Inertial Deep Orientation-Estimation and Localization
Scott Sun, Dennis Melamed, Kris Kitani
DuoMo: Dual Motion Diffusion for World-Space Human Reconstruction
Yufu Wang, Evonne Ng, Soyong Shin +8
AgentFormer: Agent-Aware Transformers for Socio-Temporal Multi-Agent Forecasting
Ye Yuan, Xinshuo Weng, Yanglan Ou +1
Learnable Spatio-Temporal Map Embeddings for Deep Inertial Localization
Dennis Melamed, Karnik Ram, Vivek Roy +1
MGpi: A Computational Model of Multiagent Group Perception and Interaction
Navyata Sanghvi, Ryo Yonetani, Kris Kitani
When We First Met: Visual-Inertial Person Localization for Co-Robot Rendezvous
Xi Sun, Xinshuo Weng, Kris Kitani
Ego-Pose Estimation and Forecasting as Real-Time PD Control
Ye Yuan, Kris Kitani
Learning Human-to-Humanoid Real-Time Whole-Body Teleoperation
Tairan He, Zhengyi Luo, Wenli Xiao +4
Unlocking Exocentric Video-Language Data for Egocentric Video Representation Learning
Zi-Yi Dou, Xitong Yang, Tushar Nagarajan +5
3D Multi-Object Tracking: A Baseline and New Evaluation Metrics
Xinshuo Weng, Jianren Wang, David Held +1
Dynamics-Regulated Kinematic Policy for Egocentric Pose Estimation
Zhengyi Luo, Ryo Hachiuma, Ye Yuan +1
CacheFlow: Fast Human Motion Prediction by Cached Normalizing Flow
Takahiro Maeda, Jinkun Cao, Norimichi Ukita +1
BFM-Zero: A Promptable Behavioral Foundation Model for Humanoid Control Using Unsupervised Reinforcement Learning
Yitang Li, Zhengyi Luo, Tonghe Zhang +10
MGF: Mixed Gaussian Flow for Diverse Trajectory Prediction
Jiahe Chen, Jinkun Cao, Dahua Lin +2
OmniH2O: Universal and Dexterous Human-to-Humanoid Whole-Body Teleoperation and Learning
Tairan He, Zhengyi Luo, Xialin He +6
SkipSR: Faster Super Resolution with Token Skipping
Rohan Choudhury, Shanchuan Lin, Jianyi Wang +6
Few-Shot Learning with Intra-Class Knowledge Transfer
Vivek Roy, Yan Xu, Yu-Xiong Wang +3
Diverse Trajectory Forecasting with Determinantal Point Processes
Ye Yuan, Kris Kitani
Ground Reaction Inertial Poser: Physics-based Human Motion Capture from Sparse IMUs and Insole Pressure Sensors
Ryosuke Hori, Jyun-Ting Song, Zhengyi Luo +4
SAM 3D Body: Robust Full-Body Human Mesh Recovery
Xitong Yang, Devansh Kukreja, Don Pinkus +11
HOIGPT: Learning Long Sequence Hand-Object Interaction with Language Models
Mingzhen Huang, Fu-Jen Chu, Bugra Tekin +10
Omnigrasp: Grasping Diverse Objects with Simulated Humanoids
Zhengyi Luo, Jinkun Cao, Sammy Christen +3
SMPLOlympics: Sports Environments for Physically Simulated Humanoids
Zhengyi Luo, Jiashun Wang, Kangni Liu +9
Online No-regret Model-Based Meta RL for Personalized Navigation
Yuda Song, Ye Yuan, Wen Sun +1
HARMONIC: A Multimodal Dataset of Assistive Human-Robot Collaboration
Benjamin A. Newman, Reuben M. Aronson, Siddartha S. Srinivasa +2
Time Will Tell: New Outlooks and A Baseline for Temporal Multi-View 3D Object Detection
Jinhyung Park, Chenfeng Xu, Shijia Yang +4
Rethinking Transformer-based Set Prediction for Object Detection
Zhiqing Sun, Shengcao Cao, Yiming Yang +1
Incremental Class Discovery for Semantic Segmentation with RGBD Sensing
Yoshikatsu Nakajima, Byeongkeun Kang, Hideo Saito +1
NovaPlan: Zero-Shot Long-Horizon Manipulation via Closed-Loop Video Language Planning
Jiahui Fu, Junyu Nan, Lingfeng Sun +5
S2GO: Streaming Sparse Gaussian Occupancy Prediction
Jinhyung Park, Yihan Hu, Chensheng Peng +3
PRECOG: PREdiction Conditioned On Goals in Visual Multi-Agent Settings
Nicholas Rhinehart, Rowan McAllister, Kris Kitani +1
GNN3DMOT: Graph Neural Network for 3D Multi-Object Tracking with Multi-Feature Learning
Xinshuo Weng, Yongxin Wang, Yunze Man +1
Harmony4D: A Video Dataset for In-The-Wild Close Human Interactions
Rawal Khirodkar, Jyun-Ting Song, Jinkun Cao +2
Joint Metrics Matter: A Better Standard for Trajectory Forecasting
Erica Weng, Hana Hoshino, Deva Ramanan +1
Lift4D: Harmonizing Single-View 3D Estimation for 4D Reconstruction In-the-Wild
Yehonathan Litman, Xiaoxuan Ma, Manan Shah +4
ExpertAF: Expert Actionable Feedback from Video
Kumar Ashutosh, Tushar Nagarajan, Georgios Pavlakos +2