193 citations · 327 across the 16 of their papers we have counts for
13 papers · 1 filter
FlexEControl: Flexible and Efficient Multimodal Control for Text-to-Image Generation
Xuehai He, Jian Zheng, Jacob Zhiyuan Fang +6
Controllable text-to-image (T2I) diffusion models generate images conditioned on both text prompts and semantic inputs of other modalities like edge maps. Nevertheless, current con…
Characterizing Video Question Answering with Sparsified Inputs
Shiyuan Huang, Robinson Piramuthu, Vicente Ordonez +2
In Video Question Answering, videos are often processed as a full-length sequence of frames to ensure minimal loss of information. Recent works have demonstrated evidence that spar…
CLIP-Nav: Using CLIP for Zero-Shot Vision-and-Language Navigation
Vishnu Sashank Dorbala, Gunnar Sigurdsson, Robinson Piramuthu +2
Household environments are visually diverse. Embodied agents performing Vision-and-Language Navigation (VLN) in the wild must be able to handle this diversity, while also following…
Video in 10 Bits: Few-Bit VideoQA for Efficiency and Privacy
Shiyuan Huang, Robinson Piramuthu, Shih-Fu Chang +1
In Video Question Answering (VideoQA), answering general questions about a video requires its visual information. Yet, video often contains redundant information irrelevant to the…
A Simple Approach for Visual Rearrangement: 3D Mapping and Semantic Search
Brandon Trabucco, Gunnar Sigurdsson, Robinson Piramuthu +2
Physically rearranging objects is an important capability for embodied agents. Visual room rearrangement evaluates an agent's ability to rearrange objects in a room to a desired go…
Visual Grounding in Video for Unsupervised Word Translation
Gunnar A. Sigurdsson, Jean-Baptiste Alayrac, Aida Nematzadeh +5
There are thousands of actively spoken languages on Earth, but a single visual world. Grounding in this visual world has the potential to bridge the gap between all these languages…
Beyond the Camera: Neural Networks in World Coordinates
Gunnar A. Sigurdsson, Abhinav Gupta, Cordelia Schmid +1
Eye movement and strategic placement of the visual field onto the retina, gives animals increased resolution of the scene and suppresses distracting information. This fundamental s…
Charades-Ego: A Large-Scale Dataset of Paired Third and First Person Videos
Gunnar A. Sigurdsson, Abhinav Gupta, Cordelia Schmid +2
In Actor and Observer we introduced a dataset linking the first and third-person video understanding domains, the Charades-Ego Dataset. In this paper we describe the egocentric asp…
Actor and Observer: Joint Modeling of First and Third-Person Videos
Gunnar A. Sigurdsson, Abhinav Gupta, Cordelia Schmid +2
Several theories in cognitive neuroscience suggest that when people interact with the world, or simulate interactions, they do so from a first-person egocentric perspective, and se…
What Actions are Needed for Understanding Human Actions in Videos?
Gunnar A. Sigurdsson, Olga Russakovsky, Abhinav Gupta
What is the right way to reason about human activities? What directions forward are most promising? In this work, we analyze the current state of human activity understanding in vi…
Asynchronous Temporal Fields for Action Recognition
Gunnar A. Sigurdsson, Santosh Divvala, Ali Farhadi +1
Actions are more than just movements and trajectories: we cook to eat and we hold a cup to drink from it. A thorough understanding of videos requires going beyond appearance modeli…
Learning Visual Storylines with Skipping Recurrent Neural Networks
Gunnar A. Sigurdsson, Xinlei Chen, Abhinav Gupta
What does a typical visit to Paris look like? Do people first take photos of the Louvre and then the Eiffel Tower? Can we visually model a temporal event like "Paris Vacation" usin…
Hollywood in Homes: Crowdsourcing Data Collection for Activity Understanding
Gunnar A. Sigurdsson, Gül Varol, Xiaolong Wang +3
Computer vision has a great potential to help our daily lives by searching for lost keys, watering flowers or reminding us to take a pill. To succeed with such tasks, computer visi…