418 citations · 518 across the 15 of their papers we have counts for
16 papers
ContextFlow: In-Context Flow Matching for Robot Manipulation
Jian Ding, Xianjie Dai, Roei Herzig +5
Although highly effective in vision and language domains, applying in-context learning to robotics remains challenging. Existing autoregressive in-context imitation methods discret…
EgoPlay: Event-Triggered Video Editing for Egocentric Streams
Jinjie Mai, Gordon Guocheng Qian, Willi Menapace +8
We introduce EgoPlay, an event-triggered video-to-video editor for egocentric streams, obtained by fine-tuning a pretrained V2V diffusion transformer on event-conditioned data buil…
EasyV2V: A High-quality Instruction-based Video Editing Framework
Jinjie Mai, Chaoyang Wang, Guocheng Gordon Qian +5
While image editing has advanced rapidly, video editing remains less explored, facing challenges in consistency, control, and generalization. We study the design space of data, arc…
Diffusion-Based Imaginative Coordination for Bimanual Manipulation
Huilin Xu, Jian Ding, Jiakun Xu +7
Bimanual manipulation is crucial in robotics, enabling complex tasks in industrial automation and household services. However, it poses significant challenges due to the high-dimen…
Can Video Diffusion Model Reconstruct 4D Geometry?
Jinjie Mai, Wenxuan Zhu, Haozhe Liu +4
Reconstructing dynamic 3D scenes (i.e., 4D geometry) from monocular video is an important yet challenging problem. Conventional multiview geometry-based approaches often struggle w…
4D-Bench: Benchmarking Multi-modal Large Language Models for 4D Object Understanding
Wenxuan Zhu, Bing Li, Cheng Zheng +8
Multimodal Large Language Models (MLLMs) have demonstrated impressive 2D image/video understanding capabilities. However, there are no publicly standardized benchmarks to assess th…
TrackNeRF: Bundle Adjusting NeRF from Sparse and Noisy Views via Feature Tracks
Jinjie Mai, Wenxuan Zhu, Sara Rojas +6
Neural radiance fields (NeRFs) generally require many images with accurate poses for accurate novel view synthesis, which does not reflect realistic setups where views can be spars…
Hybrid Structure-from-Motion and Camera Relocalization for Enhanced Egocentric Localization
Jinjie Mai, Abdullah Hamdi, Silvio Giancola +2
We built our pipeline EgoLoc-v1, mainly inspired by EgoLoc. We propose a model ensemble strategy to improve the camera pose estimation part of the VQ3D task, which has been proven…
Vivid-ZOO: Multi-View Video Generation with Diffusion Model
Bing Li, Cheng Zheng, Wenxuan Zhu +4
While diffusion models have shown impressive performance in 2D image/video generation, diffusion-based Text-to-Multi-view-Video (T2MVid) generation remains underexplored. The new c…
GES: Generalized Exponential Splatting for Efficient Radiance Field Rendering
Abdullah Hamdi, Luke Melas-Kyriazi, Jinjie Mai +5
Advancements in 3D Gaussian Splatting have significantly accelerated 3D reconstruction and generation. However, it may require a large number of Gaussians, which creates a substant…
Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priors
Guocheng Qian, Jinjie Mai, Abdullah Hamdi +8
We present Magic123, a two-stage coarse-to-fine approach for high-quality, textured 3D meshes generation from a single unposed image in the wild using both2D and 3D priors. In the…
Mindstorms in Natural Language-Based Societies of Mind
Mingchen Zhuge, Haozhe Liu, Francesco Faccio +23
Both Minsky's "society of mind" and Schmidhuber's "learning to think" inspire diverse societies of large multimodal neural networks (NNs) that solve problems by interviewing each o…
LLM as A Robotic Brain: Unifying Egocentric Memory and Control
Jinjie Mai, Jun Chen, Bing Li +3
Embodied AI focuses on the study and development of intelligent systems that possess a physical or virtual embodiment (i.e. robots) and are able to dynamically interact with their…
EgoLoc: Revisiting 3D Object Localization from Egocentric Videos with Visual Queries
Jinjie Mai, Abdullah Hamdi, Silvio Giancola +2
With the recent advances in video and 3D understanding, novel 4D spatio-temporal methods fusing both concepts have emerged. Towards this direction, the Ego4D Episodic Memory Benchm…
Estimating more camera poses for ego-centric videos is essential for VQ3D
Jinjie Mai, Chen Zhao, Abdullah Hamdi +2
Visual queries 3D localization (VQ3D) is a task in the Ego4D Episodic Memory Benchmark. Given an egocentric video, the goal is to answer queries of the form "Where did I last see o…
PointNeXt: Revisiting PointNet++ with Improved Training and Scaling Strategies
Guocheng Qian, Yuchen Li, Houwen Peng +4
PointNet++ is one of the most influential neural architectures for point cloud understanding. Although the accuracy of PointNet++ has been largely surpassed by recent networks such…