616 citations · 2k across the 27 of their papers we have counts for
30 papers
EgoCast: Forecasting Egocentric Human Pose in the Wild
Maria Escobar, Juanita Puentes, Cristhian Forigua +3
Accurately estimating and forecasting human body pose is important for enhancing the user's sense of immersion in Augmented Reality. Addressing this need, our paper introduces EgoC…
EgoCOL: Egocentric Camera pose estimation for Open-world 3D object Localization @Ego4D challenge 2023
Cristhian Forigua, Maria Escobar, Jordi Pont-Tuset +2
We present EgoCOL, an egocentric camera pose estimation method for open-world 3D object localization. Our method leverages sparse camera pose reconstructions in a two-fold manner,…
Vid2Seq: Large-Scale Pretraining of a Visual Language Model for Dense Video Captioning
Antoine Yang, Arsha Nagrani, Paul Hongsuck Seo +5
In this work, we introduce Vid2Seq, a multi-modal single-stage dense event captioning model pretrained on narrated videos which are readily-available at scale. The Vid2Seq architec…
Connecting Vision and Language with Video Localized Narratives
Paul Voigtlaender, Soravit Changpinyo, Jordi Pont-Tuset +2
We propose Video Localized Narratives, a new form of multimodal video annotations connecting vision and language. In the original Localized Narratives, annotators speak and move th…
Imagen Editor and EditBench: Advancing and Evaluating Text-Guided Image Inpainting
Su Wang, Chitwan Saharia, Ceslee Montgomery +11
Text-guided image editing can have a transformative impact in supporting creative applications. A key challenge is to generate edits that are faithful to input text prompts, while…
Two-Level Temporal Relation Model for Online Video Instance Segmentation
Çağan Selim Çoban, Oğuzhan Keskin, Jordi Pont-Tuset +1
In Video Instance Segmentation (VIS), current approaches either focus on the quality of the results, by taking the whole video as input and processing it offline; or on speed, by h…
Video Swin Transformers for Egocentric Video Understanding @ Ego4D Challenges 2022
Maria Escobar, Laura Daza, Cristina González +2
We implemented Video Swin Transformer as a base architecture for the tasks of Point-of-No-Return temporal localization and Object State Change Classification. Our method achieved c…
Crossmodal-3600: A Massively Multilingual Multimodal Evaluation Dataset
Ashish V. Thapliyal, Jordi Pont-Tuset, Xi Chen +1
Research in massively multilingual image captioning has been severely hampered by a lack of high-quality evaluation datasets. In this paper we present the Crossmodal-3600 dataset (…
Panoptic Narrative Grounding
C. González, N. Ayobi, I. Hernández +3
This paper proposes Panoptic Narrative Grounding, a spatially fine and general formulation of the natural language visual grounding problem. We establish an experimental framework…
PanGEA: The Panoramic Graph Environment Annotation Toolkit
Alexander Ku, Peter Anderson, Jordi Pont-Tuset +1
PanGEA, the Panoramic Graph Environment Annotation toolkit, is a lightweight toolkit for collecting speech and text annotations in photo-realistic 3D environments. PanGEA immerses…
Telling the What while Pointing to the Where: Multimodal Queries for Image Retrieval
Soravit Changpinyo, Jordi Pont-Tuset, Vittorio Ferrari +1
Most existing image retrieval systems use text queries as a way for the user to express what they are looking for. However, fine-grained image retrieval often requires the ability…
Connecting Vision and Language with Localized Narratives
Jordi Pont-Tuset, Jasper Uijlings, Soravit Changpinyo +2
We propose Localized Narratives, a new form of multimodal image annotations connecting vision and language. We ask annotators to describe an image with their voice while simultaneo…
Natural Vocabulary Emerges from Free-Form Annotations
Jordi Pont-Tuset, Michael Gygli, Vittorio Ferrari
We propose an approach for annotating object classes using free-form text written by undirected and untrained annotators. Free-form labeling is natural for annotators, they intuiti…
The 2019 DAVIS Challenge on VOS: Unsupervised Multi-Object Segmentation
Sergi Caelles, Jordi Pont-Tuset, Federico Perazzi +3
We present the 2019 DAVIS Challenge on Video Object Segmentation, the third edition of the DAVIS Challenge series, a public competition designed for the task of Video Object Segmen…
The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scale
Alina Kuznetsova, Hassan Rom, Neil Alldrin +9
We present Open Images V4, a dataset of 9.2M images with unified annotations for image classification, object detection and visual relationship detection. The images have a Creativ…
Iterative Deep Learning for Road Topology Extraction
Carles Ventura, Jordi Pont-Tuset, Sergi Caelles +2
This paper tackles the task of estimating the topology of road networks from aerial images. Building on top of a global model that performs a dense semantical classification of the…
Blazingly Fast Video Object Segmentation with Pixel-Wise Metric Learning
Yuhua Chen, Jordi Pont-Tuset, Alberto Montes +1
This paper tackles the problem of video object segmentation, given some user annotation which indicates the object of interest. The problem is formulated as pixel-wise retrieval in…
The 2018 DAVIS Challenge on Video Object Segmentation
Sergi Caelles, Alberto Montes, Kevis-Kokitsi Maninis +4
We present the 2018 DAVIS Challenge on Video Object Segmentation, a public competition specifically designed for the task of video object segmentation. It builds upon the DAVIS 201…
Iterative Deep Learning for Network Topology Extraction
Carles Ventura, Jordi Pont-Tuset, Sergi Caelles +2
This paper tackles the task of estimating the topology of filamentary networks such as retinal vessels and road networks. Building on top of a global model that performs a dense se…
Detection-aided liver lesion segmentation using deep learning
Miriam Bellver, Kevis-Kokitsi Maninis, Jordi Pont-Tuset +3
A fully automatic technique for segmenting the liver and localizing its unhealthy tissues is a convenient tool in order to diagnose hepatic diseases and assess the response to the…
Deep Extreme Cut: From Extreme Points to Object Segmentation
Kevis-Kokitsi Maninis, Sergi Caelles, Jordi Pont-Tuset +1
This paper explores the use of extreme points in an object (left-most, right-most, top, bottom pixels) as input to obtain precise object segmentation for images and videos. We do s…
Video Object Segmentation Without Temporal Information
Kevis-Kokitsi Maninis, Sergi Caelles, Yuhua Chen +4
Video Object Segmentation, and video processing in general, has been historically dominated by methods that rely on the temporal consistency and redundancy in consecutive video fra…
Semantically-Guided Video Object Segmentation
Sergi Caelles, Yuhua Chen, Jordi Pont-Tuset +1
This paper tackles the problem of semi-supervised video object segmentation, that is, segmenting an object in a sequence given its mask in the first frame. One of the main challeng…
The 2017 DAVIS Challenge on Video Object Segmentation
Jordi Pont-Tuset, Federico Perazzi, Sergi Caelles +3
We present the 2017 DAVIS Challenge on Video Object Segmentation, a public dataset, benchmark, and competition specifically designed for the task of video object segmentation. Foll…