93 citations · 253 across the 25 of their papers we have counts for
25 papers
Analyzing The Language of Visual Tokens
David M. Chan, Rodolfo Corona, Joonyong Park +3
With the introduction of transformer-based models for vision and language tasks, such as LLaVA and Chameleon, there has been renewed interest in the discrete tokenized representati…
Evaluating Multiview Object Consistency in Humans and Image Models
Tyler Bonnen, Stephanie Fu, Yutong Bai +5
We introduce a benchmark to directly evaluate the alignment between human observers and vision models on a 3D shape inference task. We leverage an experimental design from the cogn…
KiVA: Kid-inspired Visual Analogies for Testing Large Multimodal Models
Eunice Yiu, Maan Qraitem, Anisa Noor Majhi +5
This paper investigates visual analogical reasoning in large multimodal models (LMMs) compared to human adults and children. A "visual analogy" is an abstract rule inferred from on…
LLARVA: Vision-Action Instruction Tuning Enhances Robot Learning
Dantong Niu, Yuvan Sharma, Giscard Biamby +5
In recent years, instruction-tuned Large Multimodal Models (LMMs) have been successful at several tasks, including image captioning and visual question answering; yet leveraging th…
Finding Visual Task Vectors
Alberto Hojel, Yutong Bai, Trevor Darrell +2
Visual Prompting is a technique for teaching models to perform a visual task via in-context examples, without any additional training. In this work, we analyze the activations of M…
Sequential Modeling Enables Scalable Learning for Large Vision Models
Yutong Bai, Xinyang Geng, Karttikeya Mangalam +5
We introduce a novel sequential modeling approach which enables learning a Large Vision Model (LVM) without making use of any linguistic data. To do this, we define a common format…
Understanding Pan-Sharpening via Generalized Inverse
Shiqi Liu, Yihua Tan, Yutong Bai +1
Pan-sharpening algorithms utilize a panchromatic image and a multispectral image to generate a high spatial and high spectral image. However, the optimizations of the algorithms ar…
Discovering Failure Modes of Text-guided Diffusion Models via Adversarial Search
Qihao Liu, Adam Kortylewski, Yutong Bai +2
Text-guided diffusion models (TDMs) are widely applied but can fail unexpectedly. Common failures include: (i) natural-looking text prompts generating images with the wrong content…
Delving into Masked Autoencoders for Multi-Label Thorax Disease Classification
Junfei Xiao, Yutong Bai, Alan Yuille +1
Vision Transformer (ViT) has become one of the most popular neural architectures due to its great scalability, computational efficiency, and compelling performance in many vision t…
Making Your First Choice: To Address Cold Start Problem in Vision Active Learning
Liangyu Chen, Yutong Bai, Siyu Huang +4
Active learning promises to improve annotation efficiency by iteratively selecting the most important data to be annotated first. However, we uncover a striking contradiction to th…
Masked Autoencoders Enable Efficient Knowledge Distillers
Yutong Bai, Zeyu Wang, Junfei Xiao +5
This paper studies the potential of distilling knowledge from pre-trained models, especially Masked Autoencoders. Our approach is simple: in addition to optimizing the pixel recons…
Can CNNs Be More Robust Than Transformers?
Zeyu Wang, Yutong Bai, Yuyin Zhou +1
The recent success of Vision Transformers is shaking the long dominance of Convolutional Neural Networks (CNNs) in image recognition for a decade. Specifically, in terms of robustn…
Fast AdvProp
Jieru Mei, Yucheng Han, Yutong Bai +5
Adversarial Propagation (AdvProp) is an effective way to improve recognition models, leveraging adversarial examples. Nonetheless, AdvProp suffers from the extremely slow training…
Point-Level Region Contrast for Object Detection Pre-Training
Yutong Bai, Xinlei Chen, Alexander Kirillov +2
In this work we present point-level region contrast, a self-supervised pre-training approach for the task of object detection. This approach is motivated by the two key factors in…
Are Transformers More Robust Than CNNs?
Yutong Bai, Jieru Mei, Alan Yuille +1
Transformer emerges as a powerful tool for visual recognition. In addition to demonstrating competitive performance on a broad range of visual benchmarks, recent works also argue t…
Glance-and-Gaze Vision Transformer
Qihang Yu, Yingda Xia, Yutong Bai +3
Recently, there emerges a series of vision Transformers, which show superior performance with a more compact model size than conventional convolutional neural networks, thanks to t…
CateNorm: Categorical Normalization for Robust Medical Image Segmentation
Junfei Xiao, Lequan Yu, Zongwei Zhou +4
Batch normalization (BN) uniformly shifts and scales the activations based on the statistics of a batch of images. However, the intensity distribution of the background pixels ofte…
TransFG: A Transformer Architecture for Fine-grained Recognition
Ju He, Jie-Neng Chen, Shuai Liu +4
Fine-grained visual classification (FGVC) which aims at recognizing objects from subcategories is a very challenging task due to the inherently subtle inter-class differences. Most…
Mask Guided Matting via Progressive Refinement Network
Qihang Yu, Jianming Zhang, He Zhang +5
We propose Mask Guided (MG) Matting, a robust matting framework that takes a general coarse mask as guidance. MG Matting leverages a network (PRN) design which encourages the matti…
Unsupervised Part Discovery via Feature Alignment
Mengqi Guo, Yutong Bai, Zhishuai Zhang +2
Understanding objects in terms of their individual parts is important, because it enables a precise understanding of the objects' geometrical structure, and enhances object recogni…
Can Temporal Information Help with Contrastive Self-Supervised Learning?
Yutong Bai, Haoqi Fan, Ishan Misra +6
Leveraging temporal information has been regarded as essential for developing video understanding models. However, how to properly incorporate temporal information into the recent…
CoKe: Localized Contrastive Learning for Robust Keypoint Detection
Yutong Bai, Angtian Wang, Adam Kortylewski +1
In this paper, we introduce a contrastive learning framework for keypoint detection (CoKe). Keypoint detection differs from other visual tasks where contrastive learning has been a…
C2FNAS: Coarse-to-Fine Neural Architecture Search for 3D Medical Image Segmentation
Qihang Yu, Dong Yang, Holger Roth +4
3D convolution neural networks (CNN) have been proved very successful in parsing organs or tumours in 3D medical images, but it remains sophisticated and time-consuming to choose o…
CLEVR-Ref+: Diagnosing Visual Reasoning with Referring Expressions
Runtao Liu, Chenxi Liu, Yutong Bai +1
Referring object detection and referring image segmentation are important tasks that require joint understanding of visual information and natural language. Yet there has been evid…