7 papers
Which Way Did It Move? Diagnosing and Overcoming Directional Motion Blindness in Video-LLMs
Jongseo Lee, Hyuntak Lee, Sunghun Kim +3
Video Large Language Models (Video-LLMs) have made rapid progress on temporal video understanding, yet many fail at a basic perceptual primitive: signed image-plane motion directio…
SPIRIT: Perceptive Shared Autonomy for Robust Robotic Manipulation under Deep Learning Uncertainty
Jongseok Lee, Ribin Balachandran, Harsimran Singh +6
Deep learning (DL) has enabled impressive advances in robotic perception, yet its limited robustness and lack of interpretability hinder reliable deployment in safety critical appl…
Disentangled Concepts Speak Louder Than Words: Explainable Video Action Recognition
Jongseo Lee, Wooil Lee, Gyeong-Moon Park +2
Effective explanations of video action recognition models should disentangle how movements unfold over time from the surrounding spatial context. However, existing methods based on…
CA^2ST: Cross-Attention in Audio, Space, and Time for Holistic Video Recognition
Jongseo Lee, Joohyun Chang, Dongho Lee +1
We propose Cross-Attention in Audio, Space, and Time (CA^2ST), a transformer-based method for holistic video recognition. Recognizing actions in videos requires both spatial and te…
ESSENTIAL: Episodic and Semantic Memory Integration for Video Class-Incremental Learning
Jongseo Lee, Kyungho Bae, Kyle Min +2
In this work, we tackle the problem of video classincremental learning (VCIL). Many existing VCIL methods mitigate catastrophic forgetting by rehearsal training with a few temporal…
PCBEAR: Pose Concept Bottleneck for Explainable Action Recognition
Jongseo Lee, Wooil Lee, Gyeong-Moon Park +2
Human action recognition (HAR) has achieved impressive results with deep learning models, but their decision-making process remains opaque due to their black-box nature. Ensuring i…