8 papers
A Hierarchical Spatiotemporal Action Tokenizer for In-Context Imitation Learning in Robotics
Fawad Javed Fateh, Ali Shah Ali, Murad Popattia +4
We present a novel hierarchical spatiotemporal action tokenizer for in-context imitation learning. We first propose a hierarchical approach, which consists of two successive levels…
TemporalVLM: Video LLMs for Temporal Reasoning in Long Videos
Fawad Javed Fateh, Umer Ahmed, Hamza Khan +2
We introduce TemporalVLM, a video large language model (video LLM) for temporal reasoning and fine-grained understanding in long videos. Our approach includes a visual encoder for…
Unsupervised Skeleton-Based Action Segmentation via Hierarchical Spatiotemporal Vector Quantization
Umer Ahmed, Syed Ahmed Mahmood, Fawad Javed Fateh +3
We propose a novel hierarchical spatiotemporal vector quantization framework for unsupervised skeleton-based temporal action segmentation. We first introduce a hierarchical approac…
Joint Self-Supervised Video Alignment and Action Segmentation
Ali Shah Ali, Syed Ahmed Mahmood, Mubin Saeed +3
We introduce a novel approach for simultaneous self-supervised video alignment and action segmentation based on a unified optimal transport framework. In particular, we first tackl…
Learning by Aligning 2D Skeleton Sequences and Multi-Modality Fusion
Quoc-Huy Tran, Muhammad Ahmed, Murad Popattia +3
This paper presents a self-supervised temporal video alignment framework which is useful for several fine-grained human activity understanding applications. In contrast with the st…
Permutation-Aware Action Segmentation via Unsupervised Frame-to-Segment Alignment
Quoc-Huy Tran, Ahmed Mehmood, Muhammad Ahmed +4
This paper presents an unsupervised transformer-based framework for temporal activity segmentation which leverages not only frame-level cues but also segment-level cues. This is in…