1 citations · 6 across the 44 of their papers we have counts for
51 papers
Are We Overthinking One-Step Generation?
Woobin Park, Jaeyeon Kim, Yilun Du +2
Diffusion and flow matching are now dominant frameworks for generative modeling in continuous domains, but they typically require multiple network evaluations at inference. This ha…
InSight: A Mobile Assistant Helping Visually Impaired People Navigate Unfamiliar Environment
Junwei Zheng, Jiale Wei, Di Wen +6
Navigating unfamiliar outdoor environments is one of the most demanding everyday tasks for people with visual impairments (PVI). However, existing navigation systems cannot simulta…
VL-DocIR: A Benchmark for Vision-Based Long Document Information Retrieval
Simon Ott, Yufan Chen, Ruiping Liu +6
Vision-based document retrieval has improved rapidly, but current evaluation settings still emphasize short documents and single relevant pages. This leaves an important gap betwee…
SceneForesee: Learning Scene Evolution for Spatiotemporal Reasoning
Chaoyue Li, Shoudong Han, Haoyang Wu +4
Large multimodal models (LMMs) need to understand how entities move and interact as a scene develops. This requires following the same entities over time and relating their observe…
HEIR: Learning Human-Entity Interactions with Functional Roles
Di Wen, Wenhao Guo, Yuedong Tan +15
Understanding human-entity interactions requires recovering each person-action event's participants, roles, and shared identities. This structure can support embodied agents by cla…
Do LiDAR Language Models Really Understand Spatio-temporal Relationships?
Runyi Yang, Murat Akkoyun, Di Wen +8
Recent 4D LiDAR language models aim to reason about objects and their evolving spatial relationships. Yet, in our evaluation, always selecting the same option nearly matches the mu…
Hand-Aware Transition Modeling for Bimanual Procedural Anomaly Detection
Di Wen, Jimmy Weissert, Luc Maria Scherrer +7
Procedural anomaly detection in bimanual assembly requires judging each hand action against the execution so far. A corrective action may look unusual in isolation, while a visuall…
XLocalizer: Cross-grained Alignment for Progressive Cross-view Video Geo-localization
Zichao Zeng, Weijia Fan, Yufan Chen +7
Cross-view Video Geo-localization (CVG) aims to localize ground-view videos by retrieving their corresponding geo-tagged aerial images. However, CVG approaches rely on fixed-length…
Case2Flow: Bridging Patient Cases and Guideline Flowcharts through Multimodal Retrieval
Jiale Wei, Yufan Chen, Alexander Jaus +5
Medical guidelines encode rich, evidence-based decision logic, yet the specific decision artifact a clinician needs is hard to locate within a guideline, let alone across guideline…
GuideFetch: A Task Coordination Framework for Concurrent Navigation and Object Retrieval in Assistive Robot Dogs
Qian Yin, Ruiping Liu, Kunyu Peng +7
Consider one robot guide dog escorting a blind user to a seat while a second retrieves and delivers an object. We introduce \textsc{GuideFetch}, a framework for coordinating this c…
SeqLoc: Beyond the Single Frame for Cross-View Geo-Localization in Feature-Sparse Scenes
Junwei Zheng, Yun Huang, Ruize Dai +8
Cross-View Geo-Localization (CVGL) with OpenStreetMap (OSM) performs well in structure-rich urban environments but collapses in feature-sparse scenes such as rural roads. To study…
Seeing Together: Multi-Robot Cooperative Egocentric Spatial Reasoning with Multimodal Large Language Models
Kunyu Peng, Zhikun Zhou, Kailun Yang +9
Multimodal Large Language Models (MLLMs) have made substantial progress in egocentric video understanding, but their ability to reason cooperatively from multiple embodied viewpoin…
IMPACT-Scribe: Interactive Temporal Action Segmentation with Boundary Scribbles and Query Planning
Qian Yin, Di Wen, Kunyu Peng +11
Dense temporal annotation of procedural activity videos is vital for action understanding and embodied intelligence but remains labor-intensive due to reactive tools. Each correcti…
IMPACT-HOI: Supervisory Control for Onset-Anchored Partial HOI Event Construction
Haoshen Zhang, Di Wen, Kunyu Peng +12
We present IMPACT-HOI, a mixed-initiative framework for annotating egocentric procedural video by constructing structured event graphs for Human-Object Interactions (HOI), motivate…
EgoExoMem: Cross-View Memory Reasoning over Synchronized Egocentric and Exocentric Videos
Ruiping Liu, Junwei Zheng, Yufan Chen +7
Egocentric memory is widely used in embodied intelligence, but it may be insufficient for comprehensive spatial-temporal reasoning. Inspired by human recall from both field and obs…
IMPACT-CYCLE: A Contract-Based Multi-Agent System for Claim-Level Supervisory Correction of Long-Video Semantic Memory
Weitong Kong, Di Wen, Kunyu Peng +10
Correcting errors in long-video understanding is disproportionately costly: existing multimodal pipelines produce opaque, end-to-end outputs that expose no intermediate state for i…
IMPACT: A Dataset for Multi-Granularity Human Procedural Action Understanding in Industrial Assembly
Di Wen, Zeyun Zhong, David Schneider +18
We introduce IMPACT, a synchronized five-view RGB-D dataset for deployment-oriented industrial procedural understanding, built around real assembly and disassembly of a commercial…
Rethinking Video Human-Object Interaction: Set Prediction over Time for Unified Detection and Anticipation
Yuanhao Luo, Di Wen, Kunyu Peng +5
Video-based human-object interaction (HOI) understanding requires both detecting ongoing interactions and anticipating their future evolution. However, existing methods usually tre…
Towards Multi-Source Domain Generalization for Sleep Staging with Noisy Labels
Kening Wang, Di Wen, Yufan Chen +6
Automatic sleep staging is a multimodal learning problem involving heterogeneous physiological signals such as EEG and EOG, which often suffer from domain shifts across institution…
More than the Sum: Panorama-Language Models for Adverse Omni-Scenes
Weijia Fan, Ruiping Liu, Jiale Wei +7
Existing vision-language models (VLMs) are tailored for pinhole imagery, stitching multiple narrow field-of-view inputs to piece together a complete omni-scene understanding. Yet,…
DriveXQA: Cross-modal Visual Question Answering for Adverse Driving Scene Understanding
Mingzhe Tao, Ruiping Liu, Junwei Zheng +6
Fusing sensors with complementary modalities is crucial for maintaining a stable and comprehensive understanding of abnormal driving scenes. However, Multimodal Large Language Mode…
Not an Obstacle for Dog, but a Hazard for Human: A Co-Ego Navigation System for Guide Dog Robots
Ruiping Liu, Jingqi Zhang, Junwei Zheng +8
Guide dogs offer independence to Blind and Low-Vision (BLV) individuals, yet their limited availability leaves the vast majority of BLV users without access. Quadruped robotic guid…
-Occ: Resilient 3D Semantic Occupancy Prediction for Autonomous Driving with Incomplete Camera Inputs
Kaixin Lin, Kunyu Peng, Di Wen +3
Semantic occupancy prediction enables dense 3D geometric and semantic understanding for autonomous driving. However, existing camera-based approaches implicitly assume complete sur…
SGR3 Model: Scene Graph Retrieval-Reasoning Model in 3D
Zirui Wang, Ruiping Liu, Yufan Chen +7
3D scene graphs provide a structured representation of object entities and their relationships, enabling high-level interpretation and reasoning for robots while remaining intuitiv…