44 citations · 46 across the 7 of their papers we have counts for
8 papers
Generalizable VLA Finetuning via Representation Anchoring and Language-Action Alignment
Dwip Dalal, Shivansh Patel, Chahit Jain +8
Finetuning a pretrained vision-language model (VLM) on robot demonstrations via behavior cloning (BC) has become the standard recipe for vision-language-action (VLA) policies. Howe…
Robotic Manipulation by Imitating Generated Videos Without Physical Demonstrations
Shivansh Patel, Shraddhaa Mohan, Hanlin Mai +3
This work introduces Robots Imitating Generated Videos (RIGVid), a system that enables robots to perform complex manipulation tasks--such as pouring, wiping, and mixing--purely by…
A Real-to-Sim-to-Real Approach to Robotic Manipulation with VLM-Generated Iterative Keypoint Rewards
Shivansh Patel, Xinchen Yin, Wenlong Huang +5
Task specification for robotic manipulation in open-world environments is challenging, requiring flexible and adaptive objectives that align with human intentions and can evolve th…
Interpretation of Emergent Communication in Heterogeneous Collaborative Embodied Agents
Shivansh Patel, Saim Wani, Unnat Jain +4
Communication between embodied AI agents has received increasing attention in recent years. Despite its use, it is still unclear whether the learned communication is interpretable…
Language-Aligned Waypoint (LAW) Supervision for Vision-and-Language Navigation in Continuous Environments
Sonia Raychaudhuri, Saim Wani, Shivansh Patel +2
In the Vision-and-Language Navigation (VLN) task an embodied agent navigates a 3D environment, following natural language instructions. A challenge in this task is how to handle 'o…
MultiON: Benchmarking Semantic Map Memory using Multi-Object Navigation
Saim Wani, Shivansh Patel, Unnat Jain +2
Navigation tasks in photorealistic 3D environments are challenging because they require perception and effective planning under partial observability. Recent work shows that map-li…
Granular Multimodal Attention Networks for Visual Dialog
Badri N. Patro, Shivansh Patel, Vinay P. Namboodiri
Vision and language tasks have benefited from attention. There have been a number of different attention models proposed. However, the scale at which attention needs to be applied…
U-CAM: Visual Explanation using Uncertainty based Class Activation Maps
Badri N. Patro, Mayank Lunayach, Shivansh Patel +1
Understanding and explaining deep learning models is an imperative task. Towards this, we propose a method that obtains gradient-based certainty estimates that also provide visual…