8 citations · 14 across the 18 of their papers we have counts for
22 papers
Balancing Reference Guidance and Free Generation in Trajectory Rollouts for Reasoning RL
Hanyu Wang, Nakul Agarwal, Hossein Nourkhiz Mahjoub +4
A verified reference solution provides a correct trajectory for training a reasoning model. Alternatively, a prefix of the reference can guide the model in generating a trajectory…
Watch, Infer, Coordinate: Inferring Robot Partner Constraints for Zero-Shot Coordination
Suyu Ye, Zheyuan Zhang, Vaishnav Tadiparthi +5
Robots operating in the physical world will increasingly need to coordinate with other robots, particularly in manipulation tasks where an object may be too large or heavy for a si…
JEPA-TTT: Persistent Test-Time Training of Latent World Models for Planning under Dynamics Shifts
Zheyuan Zhang, Suyu Ye, Nakul Agarwal +5
World models enable agents to plan by predicting future states of the environment, but their predictions can become unreliable when test-time dynamics differ from those seen during…
Test-Time Unlearning via Sparse Autoencoder
Pingzhi Li, Jinhao Duan, Vaishnav Tadiparthi +6
Machine unlearning aims to remove specific knowledge from a trained large language model (LLM) without retraining from scratch. Existing methods modify model weights via gradient a…
SAGE: Synchronized Action-Gaze Recognition and Anticipation for Human Behavior Understanding
Chenyi Kuang, Nakul Agarwal
Human object interaction (HOI), gaze pattern, and their anticipation are intricately linked, providing valuable insights into cognitive processes, intentions, and behavior. However…
Generative Skill Composition for LLM Agents
Xinyu Zhao, Zhen Tan, Vaishnav Tadiparthi +5
Recent LLM agents benefit from skills for solving complex tasks. Skills encapsulate modular packages of procedural knowledge and instructions for performing specialized tasks, such…
MERGE: Guided Vision-Language Models for Multi-Actor Event Reasoning and Grounding in Human-Robot Interaction
Joerg Deigmoeller, Nakul Agarwal, Stephan Hasler +8
We introduce MERGE, a system for situational grounding of actors, objects, and events in dynamic human-robot group interactions. Effective collaboration in such settings requires c…
Towards Driver Behavior Understanding: Weakly-Supervised Risk Perception in Driving Scenes
Nakul Agarwal, Yi-Ting Chen, Behzad Dariush
Achieving zero-collision mobility remains a key objective for intelligent vehicle systems, which requires understanding driver risk perception-a complex cognitive process shaped by…
Learning Robust Reasoning through Guided Adversarial Self-Play
Shuozhe Li, Vaishnav Tadiparthi, Kwonjoon Lee +6
Reinforcement learning from verifiable rewards (RLVR) produces strong reasoning models, yet they can fail catastrophically when the conditioning context is fallible (e.g., corrupte…
CARMA: Context-Aware Situational Grounding of Human-Robot Group Interactions by Combining Vision-Language Models with Object and Action Recognition
Joerg Deigmoeller, Stephan Hasler, Nakul Agarwal +8
We introduce CARMA, a system for situational grounding in human-robot group interactions. Effective collaboration in such group settings requires situational awareness based on a c…
Overcoming Multi-step Complexity in Multimodal Theory-of-Mind Reasoning: A Scalable Bayesian Planner
Chunhui Zhang, Zhongyu Ouyang, Kwonjoon Lee +4
Theory-of-Mind (ToM) enables humans to infer mental states-such as beliefs, desires, and intentions-forming the foundation of social cognition. However, existing computational ToM…
Pose-Aware Weakly-Supervised Action Segmentation
Seth Z. Zhao, Reza Ghoddoosian, Isht Dwivedi +2
Understanding human behavior is an important problem in the pursuit of visual intelligence. A challenge in this endeavor is the extensive and costly effort required to accurately l…
ACE: Action Concept Enhancement of Video-Language Models in Procedural Videos
Reza Ghoddoosian, Nakul Agarwal, Isht Dwivedi +1
Vision-language models (VLMs) are capable of recognizing unseen actions. However, existing VLMs lack intrinsic understanding of procedural action concepts. Hence, they overfit to f…
M2D2M: Multi-Motion Generation from Text with Discrete Diffusion Models
Seunggeun Chi, Hyung-gun Chi, Hengbo Ma +4
We introduce the Multi-Motion Discrete Diffusion Models (M2D2M), a novel approach for human motion generation from textual descriptions of multiple actions, utilizing the strengths…
Can't make an Omelette without Breaking some Eggs: Plausible Action Anticipation using Large Video-Language Models
Himangi Mittal, Nakul Agarwal, Shao-Yuan Lo +1
We introduce PlausiVL, a large video-language model for anticipating action sequences that are plausible in the real-world. While significant efforts have been made towards anticip…
Vamos: Versatile Action Models for Video Understanding
Shijie Wang, Qi Zhao, Minh Quan Do +3
What makes good representations for video understanding, such as anticipating future activities, or answering video-conditioned questions? While earlier approaches focus on end-to-…
Object-centric Video Representation for Long-term Action Anticipation
Ce Zhang, Changcheng Fu, Shijie Wang +4
This paper focuses on building object-centric representations for long-term action anticipation in videos. Our key motivation is that objects provide important cues to recognize an…
Rank2Tell: A Multimodal Driving Dataset for Joint Importance Ranking and Reasoning
Enna Sachdeva, Nakul Agarwal, Suhas Chundi +5
The widespread adoption of commercial autonomous vehicles (AVs) and advanced driver assistance systems (ADAS) may largely depend on their acceptance by society, for which their per…
AntGPT: Can Large Language Models Help Long-term Action Anticipation from Videos?
Qi Zhao, Shijie Wang, Ce Zhang +5
Can we better anticipate an actor's future actions (e.g. mix eggs) by knowing what commonly happens after his/her current action (e.g. crack eggs)? What if we also know the longer-…
Weakly-Supervised Online Action Segmentation in Multi-View Instructional Videos
Reza Ghoddoosian, Isht Dwivedi, Nakul Agarwal +2
This paper addresses a new problem of weakly-supervised online action segmentation in instructional videos. We present a framework to segment streaming videos online at test time u…
Unsupervised Domain Adaptation for Spatio-Temporal Action Localization
Nakul Agarwal, Yi-Ting Chen, Behzad Dariush +1
Spatio-temporal action localization is an important problem in computer vision that involves detecting where and when activities occur, and therefore requires modeling of both spat…
Connecting Visual Experiences using Max-flow Network with Application to Visual Localization
A. H. Abdul Hafez, Nakul Agarwal, C. V. Jawahar
We are motivated by the fact that multiple representations of the environment are required to stand for the changes in appearance with time and for changes that appear in a cyclic…