17 papers
MotionBlind: Probing the Illusion of Motion Understanding in Video-LLMs
Dhairya Bhatia, Bishoy Galoaa, Oliver Fritsche +7
Video large language models (Video-LLMs) are increasingly used as the perceptual front end of world models, a role that assumes they can read motion: how fast something moves, whic…
Just Keep Prompting: Evaluating Repetitive Socratic Prompting in VLMs
Shayda Moezzi, Bishoy Galoaa, Lorena Genua +2
Deploying Vision-Language Models (VLMs) in real-world settings requires not only strong visual reasoning but also stability under sustained conversational pressure. We introduce Ju…
PanoWorld: Geometry-Consistent Panoramic Video World Modeling
Le Jiang, Xiangyu Bai, Bishoy Galoaa +7
We present PanoWorld, a panoramic video world model that generates geometry-consistent 360 video from a single image and a caption. Existing panoramic video methods optimi…
PhyGround: Benchmarking Physical Reasoning in Generative World Models
Juyi Lin, Arash Akbari, Yumei He +13
Generative world models are increasingly used for video generation, where learned simulators are expected to capture the physical rules that govern real-world dynamics. However, ev…
Human Cognition in Machines: A Unified Perspective of World Models
Timothy Rupprecht, Pu Zhao, Amir Taherin +20
This report of world models distinguishes prior works by the cognitive functions they innovate. Many works claim an almost human-like cognitive capability in their world models. To…
HORNet: Task-Guided Frame Selection for Video Question Answering with Vision-Language Models
Xiangyu Bai, Bishoy Galoaa, Sarah Ostadabbas
Video question answering (VQA) with vision-language models (VLMs) depends critically on which frames are selected from the input video, yet most systems rely on uniform or heuristi…
Motion-o: Trajectory-Grounded Video Reasoning
Bishoy Galoaa, Shayda Moezzi, Xiangyu Bai +1
Recent video reasoning models increasingly produce spatio-temporal evidence chains that localize objects at specific timestamps. While these traces improve interpretability by grou…
UniTrack: Differentiable Graph Representation Learning for Multi-Object Tracking
Bishoy Galoaa, Xiangyu Bai, Utsav Nandi +3
We present UniTrack, a plug-and-play graph-theoretic loss function designed to significantly enhance multi-object tracking (MOT) performance by directly optimizing tracking-specifi…
Structure Over Scale: Learning Visual Reasoning from Pedagogical Video
Bishoy Galoaa, Xiangyu Bai, Sarah Ostadabbas
State-of-the-art vision-language models (VLMs) score impressively on video benchmarks yet stumble on basic visual reasoning tasks involving spatial relations, navigation, and objec…
Broadening View Synthesis of Dynamic Scenes from Constrained Monocular Videos
Le Jiang, Shaotong Zhu, Yedi Luo +2
In dynamic Neural Radiance Fields (NeRF) systems, state-of-the-art novel view synthesis methods often fail under significant viewpoint deviations, producing unstable and unrealisti…
K-Track: Kalman-Enhanced Tracking for Accelerating Deep Point Trackers on Edge Devices
Bishoy Galoaa, Pau Closas, Sarah Ostadabbas
Point tracking in video sequences is a foundational capability for real-world computer vision applications, including robotics, autonomous systems, augmented reality, and video ana…
Lang2Motion: Bridging Language and Motion through Joint Embedding Spaces
Bishoy Galoaa, Xiangyu Bai, Sarah Ostadabbas
We present Lang2Motion, a framework for language-guided point trajectory generation by aligning motion manifolds with joint embedding spaces. Unlike prior work focusing on human mo…
MoReGen: Multi-Agent Motion-Reasoning Engine for Code-based Text-to-Video Synthesis
Xiangyu Bai, He Liang, Bishoy Galoaa +4
While text-to-video (T2V) generation has achieved remarkable progress in photorealism, generating intent-aligned videos that faithfully obey physics principles remains a core chall…
Look Around and Pay Attention: Multi-camera Point Tracking Reimagined with Transformers
Bishoy Galoaa, Xiangyu Bai, Shayda Moezzi +4
This paper presents LAPA (Look Around and Pay Attention), a novel end-to-end transformer-based architecture for multi-camera point tracking that integrates appearance-based matchin…
Track and Caption Any Motion: Open-Vocabulary Spatiotemporal Captioning via Trajectory-Conditioned Generation
Bishoy Galoaa, Sarah Ostadabbas
We present TCAM (Track and Caption Any Motion), a generative framework that watches video without a text query or region prompt and produces open-vocabulary captions, temporal inte…
Learning Multimodal AI Algorithms for Amplifying Limited User Input into High-dimensional Control Space
Ali Rabiee, Sima Ghafoori, MH Farhadi +5
Current invasive assistive technologies are designed to infer high-dimensional motor control signals from severely paralyzed patients. However, they face significant challenges, in…
STREAMS: An Assistive Multimodal AI Framework for Empowering Biosignal Based Robotic Controls
Ali Rabiee, Sima Ghafoori, Xiangyu Bai +2
End-effector based assistive robots face persistent challenges in generating smooth and robust trajectories when controlled by human's noisy and unreliable biosignals such as muscl…