164 citations · 225 across the 28 of their papers we have counts for
29 papers
Learning to Taste: A Multimodal Wine Dataset
Thoranna Bender, Simon Moe Sørensen, Alireza Kashani +5
We present WineSensed, a large multimodal wine dataset for studying the relations between visual perception, language, and flavor. The dataset encompasses 897k images of wine label…
Fashionpedia-Ads: Do Your Favorite Advertisements Reveal Your Fashion Taste?
Mengyun Shi, Claire Cardie, Serge Belongie
Consumers are exposed to advertisements across many different domains on the internet, such as fashion, beauty, car, food, and others. On the other hand, fashion represents second…
Fashionpedia-Taste: A Dataset towards Explaining Human Fashion Taste
Mengyun Shi, Serge Belongie, Claire Cardie
Existing fashion datasets do not consider the multi-facts that cause a consumer to like or dislike a fashion image. Even two consumers like a same fashion image, they could like th…
Discriminative Class Tokens for Text-to-Image Diffusion Models
Idan Schwartz, Vésteinn Snæbjarnarson, Hila Chefer +4
Recent advances in text-to-image diffusion models have enabled the generation of diverse and high-quality images. While impressive, the images often fall short of depicting subtle…
Re-evaluating the Need for Multimodal Signals in Unsupervised Grammar Induction
Boyi Li, Rodolfo Corona, Karttikeya Mangalam +7
Are multimodal inputs necessary for grammar induction? Recent work has shown that multimodal training inputs can improve grammar induction. However, these improvements are based on…
PyTorch Adapt
Kevin Musgrave, Serge Belongie, Ser-Nam Lim
PyTorch Adapt is a library for domain adaptation, a type of machine learning algorithm that re-purposes existing models to work in new domains. It is a fully-featured toolkit, allo…
Assessing Neural Network Robustness via Adversarial Pivotal Tuning
Peter Ebert Christensen, Vésteinn Snæbjarnarson, Andrea Dittadi +2
The robustness of image classifiers is essential to their deployment in the real world. The ability to assess this resilience to manipulations or deviations from the training data…
Searching for Structure in Unfalsifiable Claims
Peter Ebert Christensen, Frederik Warburg, Menglin Jia +1
Social media platforms give rise to an abundance of posts and comments on every topic imaginable. Many of these posts express opinions on various aspects of society, but their unfa…
Three New Validators and a Large-Scale Benchmark Ranking for Unsupervised Domain Adaptation
Kevin Musgrave, Serge Belongie, Ser-Nam Lim
Changes to hyperparameters can have a dramatic effect on model accuracy. Thus, the tuning of hyperparameters plays an important role in optimizing machine-learning models. An integ…
Exploring Fine-Grained Audiovisual Categorization with the SSW60 Dataset
Grant Van Horn, Rui Qian, Kimberly Wilber +3
We present a new benchmark dataset, Sapsucker Woods 60 (SSW60), for advancing research on audiovisual fine-grained categorization. While our community has made great strides in fin…
On Label Granularity and Object Localization
Elijah Cole, Kimberly Wilber, Grant Van Horn +6
Weakly supervised object localization (WSOL) aims to learn representations that encode object location using only image-level category labels. However, many objects can be labeled…
Multimodal Open-Vocabulary Video Classification via Pre-Trained Vision and Language Models
Rui Qian, Yeqing Li, Zheng Xu +3
Utilizing vision and language models (VLMs) pre-trained on large-scale image-text pairs is becoming a promising paradigm for open-vocabulary visual recognition. In this work, we ex…
Text-Driven Stylization of Video Objects
Sebastian Loeschcke, Serge Belongie, Sagie Benaim
We tackle the task of stylizing video objects in an intuitive and semantic manner following a user-specified text prompt. This is a challenging task as the resulting video must sat…
Volumetric Disentanglement for 3D Scene Manipulation
Sagie Benaim, Frederik Warburg, Peter Ebert Christensen +1
Recently, advances in differential volumetric rendering enabled significant breakthroughs in the photo-realistic and fine-detailed reconstruction of complex 3D scenes, which is key…
Visual Prompt Tuning
Menglin Jia, Luming Tang, Bor-Chun Chen +4
The current modus operandi in adapting pre-trained models involves updating all the backbone parameters, ie, full fine-tuning. This paper introduces Visual Prompt Tuning (VPT) as a…
Residual Aligned: Gradient Optimization for Non-Negative Image Synthesis
Flora Yu Shen, Katie Luo, Guandao Yang +2
In this work, we address an important problem of optical see through (OST) augmented reality: non-negative image synthesis. Most of the image generation methods fail under this con…
Stay Positive: Non-Negative Image Synthesis for Augmented Reality
Katie Luo, Guandao Yang, Wenqi Xian +3
In applications such as optical see-through and projector augmented reality, producing images amounts to solving non-negative image generation, where one can only add light to an e…
Language-driven Semantic Segmentation
Boyi Li, Kilian Q. Weinberger, Serge Belongie +2
We present LSeg, a novel model for language-driven semantic image segmentation. LSeg uses a text encoder to compute embeddings of descriptive input labels (e.g., "grass" or "buildi…
Rethinking Nearest Neighbors for Visual Classification
Menglin Jia, Bor-Chun Chen, Zuxuan Wu +3
Neural network classifiers have become the de-facto choice for current "pre-train then fine-tune" paradigms of visual classification. In this paper, we investigate k-Nearest-Neighb…
Exploring Temporal Granularity in Self-Supervised Video Representation Learning
Rui Qian, Yeqing Li, Liangzhe Yuan +7
This work presents a self-supervised learning framework named TeG to explore Temporal Granularity in learning video representations. In TeG, we sample a long clip from a video and…
Unsupervised Domain Adaptation: A Reality Check
Kevin Musgrave, Serge Belongie, Ser-Nam Lim
Interest in unsupervised domain adaptation (UDA) has surged in recent years, resulting in a plethora of new algorithms. However, as is often the case in fast-moving fields, baselin…
Fine-Grained Image Analysis with Deep Learning: A Survey
Xiu-Shen Wei, Yi-Zhe Song, Oisin Mac Aodha +5
Fine-grained image analysis (FGIA) is a longstanding and fundamental problem in computer vision and pattern recognition, and underpins a diverse set of real-world applications. The…
Occluded Video Instance Segmentation: Dataset and ICCV 2021 Challenge
Jiyang Qi, Yan Gao, Yao Hu +7
Although deep learning methods have achieved advanced video object recognition performance in recent years, perceiving heavily occluded objects in a video is still a very challengi…
Robustness and Generalization via Generative Adversarial Training
Omid Poursaeed, Tianxing Jiang, Harry Yang +2
While deep neural networks have achieved remarkable success in various computer vision tasks, they often fail to generalize to new domains and subtle variations of input images. Se…