53 citations · 78 across the 11 of their papers we have counts for
12 papers
ThermEval: A Structured Benchmark for Evaluation of Vision-Language Models on Thermal Imagery
Ayush Shrivastava, Kirtan Gangani, Laksh Jain +2
Vision language models (VLMs) achieve strong performance on RGB imagery, but they do not generalize to thermal images. Thermal sensing plays a critical role in settings where visib…
Point Prompting: Counterfactual Tracking with Video Diffusion Models
Ayush Shrivastava, Sanyam Mehta, Daniel Geng +1
Trackers and video generators solve closely related problems: the former analyze motion, while the latter synthesize it. We show that this connection enables pretrained video diffu…
Fine-grained Defocus Blur Control for Generative Image Models
Ayush Shrivastava, Connelly Barnes, Xuaner Zhang +4
Current text-to-image diffusion models excel at generating diverse, high-quality images, yet they struggle to incorporate fine-grained camera metadata such as precise aperture sett…
Self-Supervised Spatial Correspondence Across Modalities
Ayush Shrivastava, Andrew Owens
We present a method for finding cross-modal space-time correspondences. Given two images from different visual modalities, such as an RGB image and a depth map, our model identifie…
Towards Unbiased and Robust Spatio-Temporal Scene Graph Generation and Anticipation
Rohith Peddi, Saurabh, Ayush Abhay Shrivastava +2
Spatio-Temporal Scene Graphs (STSGs) provide a concise and expressive representation of dynamic scenes by modeling objects and their evolving relationships over time. However, real…
Self-Supervised Any-Point Tracking by Contrastive Random Walks
Ayush Shrivastava, Andrew Owens
We present a simple, self-supervised approach to the Tracking Any Point (TAP) problem. We train a global matching transformer to find cycle consistent tracks through video via cont…
EXIF as Language: Learning Cross-Modal Associations Between Images and Camera Metadata
Chenhao Zheng, Ayush Shrivastava, Andrew Owens
We learn a visual representation that captures information about the camera that recorded a given photo. To do this, we train a multimodal embedding between image patches and the E…
TEACh: Task-driven Embodied Agents that Chat
Aishwarya Padmakumar, Jesse Thomason, Ayush Shrivastava +6
Robots operating in human spaces must be able to engage in natural language interaction with people, both understanding and executing instructions, and using conversation to resolv…
VISITRON: Visual Semantics-Aligned Interactively Trained Object-Navigator
Ayush Shrivastava, Karthik Gopalakrishnan, Yang Liu +4
Interactive robots navigating photo-realistic environments need to be trained to effectively leverage and handle the dynamic nature of dialogue in addition to the challenges underl…
Sim-to-Real Transfer for Vision-and-Language Navigation
Peter Anderson, Ayush Shrivastava, Joanne Truong +4
We study the challenging problem of releasing a robot in a previously unseen environment, and having it follow unconstrained natural language navigation instructions. Recent work o…
Improving Vision-and-Language Navigation with Image-Text Pairs from the Web
Arjun Majumdar, Ayush Shrivastava, Stefan Lee +3
Following a navigation instruction such as 'Walk down the stairs and stop at the brown sofa' requires embodied AI agents to ground scene elements referenced via language (e.g. 'sta…
Chasing Ghosts: Instruction Following as Bayesian State Tracking
Peter Anderson, Ayush Shrivastava, Devi Parikh +2
A visually-grounded navigation instruction can be interpreted as a sequence of expected observations and actions an agent following the correct trajectory would encounter and perfo…