27 citations · 80 across the 96 of their papers we have counts for
29 papers · 2 filters
OmniAct3D: Leveraging Foundation Geometry and Evidence-Grounded Reasoning for Panoramic 3D Detection
Runtong Wu, Fei Teng, Di Wen +3
Accurate 3D detection is essential for mobile embodied agents, while Vision Foundation Models (VFMs) offer transferable visual and geometric priors. Yet existing VFM-based 3D detec…
FUSEye: Training-Light Fisheye Detection with Overlapping Views and Zero-Initialized Adapters
Wenya Su, Kai Luo, Di Wen +5
Fisheye cameras give mobile robots a single-sensor, low-cost view of their surroundings, yet the COCO-pretrained detectors that practitioners routinely reuse fail on them: strong r…
Does Local Video Understanding Transfer Across Encounters? The EgoGears Benchmark
Yuedong Tan, Lei Qi, Yu Liu +13
Embodied systems must make knowledge acquired during one encounter usable in another despite changes in viewpoint, motion, and illumination. Yet aggregate cross-video accuracy conf…
HEIR: Learning Human-Entity Interactions with Functional Roles
Di Wen, Wenhao Guo, Yuedong Tan +15
Understanding human-entity interactions requires recovering each person-action event's participants, roles, and shared identities. This structure can support embodied agents by cla…
Do LiDAR Language Models Really Understand Spatio-temporal Relationships?
Runyi Yang, Murat Akkoyun, Di Wen +8
Recent 4D LiDAR language models aim to reason about objects and their evolving spatial relationships. Yet, in our evaluation, always selecting the same option nearly matches the mu…
Algebraic Consistency Alone Does Not Certify Temporal Structure in Latent Action Models
Di Wen, Ruodi Zhang, Kailun Yang +1
Latent action models infer a code for the transition between two frames of action-free video. Recent methods regularise this code to compose additively and reverse antisymmetricall…
Hand-Aware Transition Modeling for Bimanual Procedural Anomaly Detection
Di Wen, Jimmy Weissert, Luc Maria Scherrer +7
Procedural anomaly detection in bimanual assembly requires judging each hand action against the execution so far. A corrective action may look unusual in isolation, while a visuall…
PROVIA: Procedure State Tracking for Online Mistake Detection in Egocentric Videos
Di Wen, Kailun Yang, Jimmy Weissert +7
An assistant watching egocentric video should notice a mistake from past frames alone, before the next step begins, and keep working once the person recovers. A mistake changes the…
XLocalizer: Cross-grained Alignment for Progressive Cross-view Video Geo-localization
Zichao Zeng, Weijia Fan, Yufan Chen +7
Cross-view Video Geo-localization (CVG) aims to localize ground-view videos by retrieving their corresponding geo-tagged aerial images. However, CVG approaches rely on fixed-length…
SeqLoc: Beyond the Single Frame for Cross-View Geo-Localization in Feature-Sparse Scenes
Junwei Zheng, Yun Huang, Ruize Dai +8
Cross-View Geo-Localization (CVGL) with OpenStreetMap (OSM) performs well in structure-rich urban environments but collapses in feature-sparse scenes such as rural roads. To study…
The First EgoCross Challenge at EgoVis 2026: Cross-Domain Egocentric Video Question Answering
Yuqian Fu, Tianwen Qian, Yanjun Li +30
EgoCross is a cross-domain egocentric video question answering benchmark designed to evaluate whether multimodal large language models can generalize beyond common daily-life scena…
HGeo-TopoMap: Boosting Topological Mapping with Hierarchical Geometric Priors
Siyu Li, Kunyu Peng, Di Wen +3
Topological maps are key outputs of autonomous driving perception systems, delivering essential road information for path planning. They identify instances such as centerlines and…
PS-MOT: Cultivating Instance Awareness from Point Seeds for Multi-Object Tracking
Kai Luo, Fei Teng, Mengfei Duan +6
We introduce Point-supervised Multi-Object Tracking (PS-MOT) as a cost-effective alternative to traditional bounding box supervision, shifting the focus from spatial fitting to top…
Seeing Together: Multi-Robot Cooperative Egocentric Spatial Reasoning with Multimodal Large Language Models
Kunyu Peng, Zhikun Zhou, Kailun Yang +9
Multimodal Large Language Models (MLLMs) have made substantial progress in egocentric video understanding, but their ability to reason cooperatively from multiple embodied viewpoin…
IMPACT-Scribe: Interactive Temporal Action Segmentation with Boundary Scribbles and Query Planning
Qian Yin, Di Wen, Kunyu Peng +11
Dense temporal annotation of procedural activity videos is vital for action understanding and embodied intelligence but remains labor-intensive due to reactive tools. Each correcti…
IMPACT-HOI: Supervisory Control for Onset-Anchored Partial HOI Event Construction
Haoshen Zhang, Di Wen, Kunyu Peng +12
We present IMPACT-HOI, a mixed-initiative framework for annotating egocentric procedural video by constructing structured event graphs for Human-Object Interactions (HOI), motivate…
EgoExoMem: Cross-View Memory Reasoning over Synchronized Egocentric and Exocentric Videos
Ruiping Liu, Junwei Zheng, Yufan Chen +7
Egocentric memory is widely used in embodied intelligence, but it may be insufficient for comprehensive spatial-temporal reasoning. Inspired by human recall from both field and obs…
IMPACT-CYCLE: A Contract-Based Multi-Agent System for Claim-Level Supervisory Correction of Long-Video Semantic Memory
Weitong Kong, Di Wen, Kunyu Peng +10
Correcting errors in long-video understanding is disproportionately costly: existing multimodal pipelines produce opaque, end-to-end outputs that expose no intermediate state for i…
IMPACT: A Dataset for Multi-Granularity Human Procedural Action Understanding in Industrial Assembly
Di Wen, Zeyun Zhong, David Schneider +18
We introduce IMPACT, a synchronized five-view RGB-D dataset for deployment-oriented industrial procedural understanding, built around real assembly and disassembly of a commercial…
Rethinking Video Human-Object Interaction: Set Prediction over Time for Unified Detection and Anticipation
Yuanhao Luo, Di Wen, Kunyu Peng +5
Video-based human-object interaction (HOI) understanding requires both detecting ongoing interactions and anticipating their future evolution. However, existing methods usually tre…
ProOOD: Prototype-Guided Out-of-Distribution 3D Occupancy Prediction
Yuheng Zhang, Mengfei Duan, Kunyu Peng +5
3D semantic occupancy prediction is central to autonomous driving, yet current methods are vulnerable to long-tailed class bias and out-of-distribution (OOD) inputs, often overconf…
Seeing Beyond: Extrapolative Domain Adaptive Panoramic Segmentation
Yuanfan Zheng, Kunyu Peng, Xu Zheng +1
Cross-domain panoramic semantic segmentation has attracted growing interest as it enables comprehensive 360° scene understanding for real-world applications. However, it remains pa…
-Occ: Resilient 3D Semantic Occupancy Prediction for Autonomous Driving with Incomplete Camera Inputs
Kaixin Lin, Kunyu Peng, Di Wen +3
Semantic occupancy prediction enables dense 3D geometric and semantic understanding for autonomous driving. However, existing camera-based approaches implicitly assume complete sur…
SGR3 Model: Scene Graph Retrieval-Reasoning Model in 3D
Zirui Wang, Ruiping Liu, Yufan Chen +7
3D scene graphs provide a structured representation of object entities and their relationships, enabling high-level interpretation and reasoning for robots while remaining intuitiv…