2 citations · 6 across the 14 of their papers we have counts for
15 papers
StreamTTT: Reconciling Real-Time Perception and Long-Term Memory in Streaming VLMs
Joya Chen, Zeyun Zhong, Mike Zheng Shou
Humans effortlessly perceive the present while remembering the past, yet streaming VLMs often trade off real-time perception against long-term memory. Prior work shows that shorten…
Rethinking Expressivity and Efficiency in Test-Time Training
Zeyun Zhong, Joya Chen, Manuel Martin +3
Test-Time Training (TTT) enables long-context processing via continuous weight updates during inference, but current methods struggle to balance the expressivity of per-token updat…
Multi-modal Video Representation Alignment for Robust Self-supervised Driver Distraction Detection
David J. Lerch, Livien Majer, Zeyun Zhong +3
Robust self-supervised learning of multi-modal video representations is critical for real-world applications such as driver distraction detection, where multiple sensors provide co…
FlowNar: Scalable Streaming Narration for Long-Form Videos
Zeyun Zhong, Manuel Martin, Chengzhi Wu +4
Recent Large Multimodal Models (LMMs), primarily designed for offline settings, are ill-suited for the dynamic requirements of streaming video. While recent online adaptations impr…
IMPACT-Scribe: Interactive Temporal Action Segmentation with Boundary Scribbles and Query Planning
Qian Yin, Di Wen, Kunyu Peng +11
Dense temporal annotation of procedural activity videos is vital for action understanding and embodied intelligence but remains labor-intensive due to reactive tools. Each correcti…
IMPACT-HOI: Supervisory Control for Onset-Anchored Partial HOI Event Construction
Haoshen Zhang, Di Wen, Kunyu Peng +12
We present IMPACT-HOI, a mixed-initiative framework for annotating egocentric procedural video by constructing structured event graphs for Human-Object Interactions (HOI), motivate…
IMPACT-CYCLE: A Contract-Based Multi-Agent System for Claim-Level Supervisory Correction of Long-Video Semantic Memory
Weitong Kong, Di Wen, Kunyu Peng +10
Correcting errors in long-video understanding is disproportionately costly: existing multimodal pipelines produce opaque, end-to-end outputs that expose no intermediate state for i…
IMPACT: A Dataset for Multi-Granularity Human Procedural Action Understanding in Industrial Assembly
Di Wen, Zeyun Zhong, David Schneider +18
We introduce IMPACT, a synchronized five-view RGB-D dataset for deployment-oriented industrial procedural understanding, built around real assembly and disassembly of a commercial…
SAMBLE: Shape-Specific Point Cloud Sampling for an Optimal Trade-Off Between Local Detail and Global Uniformity
Chengzhi Wu, Yuxin Wan, Hao Fu +5
Driven by the increasing demand for accurate and efficient representation of 3D data in various domains, point cloud sampling has emerged as a pivotal research topic in 3D computer…
OmniFall: From Staged Through Synthetic to Wild, A Unified Multi-Domain Dataset for Robust Fall Detection
David Schneider, Zdravko Marinov, Moritz Mistol +6
Visual fall detection models are usually trained on small, staged datasets. Their real-world utility remains unclear; such data lacks diversity and evaluation protocols differ from…
Rethinking Attention Module Design for Point Cloud Analysis
Chengzhi Wu, Kaige Wang, Zeyun Zhong +5
In recent years, there have been significant advancements in applying attention mechanisms to point cloud analysis. However, attention module variants featured in various research…
QueryMamba: A Mamba-Based Encoder-Decoder Architecture with a Statistical Verb-Noun Interaction Module for Video Action Forecasting @ Ego4D Long-Term Action Anticipation Challenge 2024
Zeyun Zhong, Manuel Martin, Frederik Diederichs +1
This report presents a novel Mamba-based encoder-decoder architecture, QueryMamba, featuring an integrated verb-noun interaction module that utilizes a statistical verb-noun co-occ…
DiffAnt: Diffusion Models for Action Anticipation
Zeyun Zhong, Chengzhi Wu, Manuel Martin +3
Anticipating future actions is inherently uncertain. Given an observed video segment containing ongoing actions, multiple subsequent actions can plausibly follow. This uncertainty…
A Survey on Deep Learning Techniques for Action Anticipation
Zeyun Zhong, Manuel Martin, Michael Voit +2
The ability to anticipate possible future human actions is essential for a wide range of applications, including autonomous driving and human-robot interaction. Consequently, numer…
Anticipative Feature Fusion Transformer for Multi-Modal Action Anticipation
Zeyun Zhong, David Schneider, Michael Voit +2
Although human action anticipation is a task which is inherently multi-modal, state-of-the-art methods on well known action anticipation datasets leverage this data by applying ens…