196 citations · 458 across the 30 of their papers we have counts for
8 papers · 1 filter
Rethinking Video ViTs: Sparse Video Tubes for Joint Image and Video Learning
AJ Piergiovanni, Weicheng Kuo, Anelia Angelova
We present a simple approach which can turn a ViT encoder into an efficient video model, which can seamlessly work with both image and video inputs. By sparsely sampling the inputs…
Compound Tokens: Channel Fusion for Vision-Language Representation Learning
Maxwell Mbabilla Aladago, AJ Piergiovanni
We present an effective method for fusing visual-and-language representations for several question answering tasks including visual question answering and visual entailment. In con…
Pre-training image-language transformers for open-vocabulary tasks
AJ Piergiovanni, Weicheng Kuo, Anelia Angelova
We present a pre-training approach for vision and language transformer models, which is based on a mixture of diverse tasks. We explore both the use of image-text captioning data i…
F-VLM: Open-Vocabulary Object Detection upon Frozen Vision and Language Models
Weicheng Kuo, Yin Cui, Xiuye Gu +2
We present F-VLM, a simple open-vocabulary object detection method built upon Frozen Vision and Language Models. F-VLM simplifies the current multi-stage training pipeline by elimi…
PaLI: A Jointly-Scaled Multilingual Language-Image Model
Xi Chen, Xiao Wang, Soravit Changpinyo +26
Effective scaling and a flexible task interface enable large language models to excel at many tasks. We present PaLI (Pathways Language and Image model), a model that extends this…
Video Question Answering with Iterative Video-Text Co-Tokenization
AJ Piergiovanni, Kairo Morton, Weicheng Kuo +2
Video question answering is a challenging task that requires understanding jointly the language input, the visual information in individual video frames, as well as the temporal in…