10 citations · 18 across the 14 of their papers we have counts for
16 papers
MERGE: Guided Vision-Language Models for Multi-Actor Event Reasoning and Grounding in Human-Robot Interaction
Joerg Deigmoeller, Nakul Agarwal, Stephan Hasler +8
We introduce MERGE, a system for situational grounding of actors, objects, and events in dynamic human-robot group interactions. Effective collaboration in such settings requires c…
Towards Driver Behavior Understanding: Weakly-Supervised Risk Perception in Driving Scenes
Nakul Agarwal, Yi-Ting Chen, Behzad Dariush
Achieving zero-collision mobility remains a key objective for intelligent vehicle systems, which requires understanding driver risk perception-a complex cognitive process shaped by…
Task-Aware Resolution Optimization for Visual Large Language Models
Weiqing Luo, Zhen Tan, Yifan Li +4
Real-world vision-language applications demand varying levels of perceptual granularity. However, most existing visual large language models (VLLMs), such as LLaVA, pre-assume a fi…
CARMA: Context-Aware Situational Grounding of Human-Robot Group Interactions by Combining Vision-Language Models with Object and Action Recognition
Joerg Deigmoeller, Stephan Hasler, Nakul Agarwal +8
We introduce CARMA, a system for situational grounding in human-robot group interactions. Effective collaboration in such group settings requires situational awareness based on a c…
Pose-Aware Weakly-Supervised Action Segmentation
Seth Z. Zhao, Reza Ghoddoosian, Isht Dwivedi +2
Understanding human behavior is an important problem in the pursuit of visual intelligence. A challenge in this endeavor is the extensive and costly effort required to accurately l…
Generalized Mission Planning for Heterogeneous Multi-Robot Teams via LLM-constructed Hierarchical Trees
Piyush Gupta, David Isele, Enna Sachdeva +4
We present a novel mission-planning strategy for heterogeneous multi-robot teams, taking into account the specific constraints and capabilities of each robot. Our approach employs…
Optimal Driver Warning Generation in Dynamic Driving Environment
Chenran Li, Aolin Xu, Enna Sachdeva +2
The driver warning system that alerts the human driver about potential risks during driving is a key feature of an advanced driver assistance system. Existing driver warning techno…
Constrained Human-AI Cooperation: An Inclusive Embodied Social Intelligence Challenge
Weihua Du, Qiushi Lyu, Jiaming Shan +8
We introduce Constrained Human-AI Cooperation (CHAIC), an inclusive embodied social intelligence challenge designed to test social perception and cooperation in embodied agents. In…
Follow the Rules: Reasoning for Video Anomaly Detection with Large Language Models
Yuchen Yang, Kwonjoon Lee, Behzad Dariush +2
Video Anomaly Detection (VAD) is crucial for applications such as security surveillance and autonomous driving. However, existing VAD methods provide little rationale behind detect…
Rank2Tell: A Multimodal Driving Dataset for Joint Importance Ranking and Reasoning
Enna Sachdeva, Nakul Agarwal, Suhas Chundi +5
The widespread adoption of commercial autonomous vehicles (AVs) and advanced driver assistance systems (ADAS) may largely depend on their acceptance by society, for which their per…
Weakly-Supervised Online Action Segmentation in Multi-View Instructional Videos
Reza Ghoddoosian, Isht Dwivedi, Nakul Agarwal +2
This paper addresses a new problem of weakly-supervised online action segmentation in instructional videos. We present a framework to segment streaming videos online at test time u…
Social-STAGE: Spatio-Temporal Multi-Modal Future Trajectory Forecast
Srikanth Malla, Chiho Choi, Behzad Dariush
This paper considers the problem of multi-modal future trajectory forecast with ranking. Here, multi-modality and ranking refer to the multiple plausible path predictions and the c…
Unsupervised Domain Adaptation for Spatio-Temporal Action Localization
Nakul Agarwal, Yi-Ting Chen, Behzad Dariush +1
Spatio-temporal action localization is an important problem in computer vision that involves detecting where and when activities occur, and therefore requires modeling of both spat…
Recognition and 3D Localization of Pedestrian Actions from Monocular Video
Jun Hayakawa, Behzad Dariush
Understanding and predicting pedestrian behavior is an important and challenging area of research for realizing safe and effective navigation strategies in automated and advanced d…
Ego-motion and Surrounding Vehicle State Estimation Using a Monocular Camera
Jun Hayakawa, Behzad Dariush
Understanding ego-motion and surrounding vehicle state is essential to enable automated driving and advanced driving assistance technologies. Typical approaches to solve this probl…
SSP: Single Shot Future Trajectory Prediction
Isht Dwivedi, Srikanth Malla, Behzad Dariush +1
We propose a robust solution to future trajectory forecast, which can be practically applicable to autonomous agents in highly crowded environments. For this, three aspects are par…