192 citations · 617 across the 27 of their papers we have counts for
28 papers
Image Generators are Generalist Vision Learners
Valentin Gabeur, Shangbang Long, Songyou Peng +22
Recent works show that image and video generators exhibit zero-shot visual understanding behaviors, in a way reminiscent of how LLMs develop emergent capabilities of language under…
RoboDreamer: Learning Compositional World Models for Robot Imagination
Siyuan Zhou, Yilun Du, Jiaben Chen +3
Text-to-video models have demonstrated substantial potential in robotic decision-making, enabling the imagination of realistic plans of future actions as well as accurate environme…
Perturbing Attention Gives You More Bang for the Buck: Subtle Imaging Perturbations That Efficiently Fool Customized Diffusion Models
Jingyao Xu, Yuetong Lu, Yandong Li +3
Diffusion models (DMs) embark a new era of generative modeling and offer more opportunities for efficient generating high-quality and realistic data samples. However, their widespr…
Instruct-Imagen: Image Generation with Multi-modal Instruction
Hexiang Hu, Kelvin C. K. Chan, Yu-Chuan Su +9
This paper presents instruct-imagen, a model that tackles heterogeneous image generation tasks and generalizes across unseen tasks. We introduce *multi-modal instruction* for image…
DreamInpainter: Text-Guided Subject-Driven Image Inpainting with Diffusion Models
Shaoan Xie, Yang Zhao, Zhisheng Xiao +5
This study introduces Text-Guided Subject-Driven Image Inpainting, a novel task that combines text and exemplar images for image inpainting. While both text and exemplar images hav…
Identity Encoder for Personalized Diffusion
Yu-Chuan Su, Kelvin C. K. Chan, Yandong Li +5
Many applications can benefit from personalized image generation models, including image enhancement, video conferences, just to name a few. Existing works achieved personalization…
Taming Encoder for Zero Fine-tuning Image Customization with Text-to-Image Diffusion Models
Xuhui Jia, Yang Zhao, Kelvin C. K. Chan +6
This paper proposes a method for generating images of customized objects specified by users. The method is based on a general framework that bypasses the lengthy optimization requi…
What's in a Name? Beyond Class Indices for Image Recognition
Kai Han, Xiaohu Huang, Yandong Li +3
Existing machine learning models demonstrate excellent performance in image object recognition after training on a large-scale dataset under full supervision. However, these models…
Subject-driven Text-to-Image Generation via Apprenticeship Learning
Wenhu Chen, Hexiang Hu, Yandong Li +4
Recent text-to-image generation models like DreamBooth have made remarkable progress in generating highly customized images of a target subject, by fine-tuning an ``expert model''…
Learning to Adapt to Online Streams with Distribution Shifts
Chenyan Wu, Yimu Pan, Yandong Li +1
Test-time adaptation (TTA) is a technique used to reduce distribution gaps between the training and testing sets by leveraging unlabeled test data during inference. In this work, w…
MUG: Multi-human Graph Network for 3D Mesh Reconstruction from 2D Pose
Chenyan Wu, Yandong Li, Xianfeng Tang +1
Reconstructing multi-human body mesh from a single monocular image is an important but challenging computer vision problem. In addition to the individual body mesh models, we need…
On Model Calibration for Long-Tailed Object Detection and Instance Segmentation
Tai-Yu Pan, Cheng Zhang, Yandong Li +5
Vanilla models for object detection and instance segmentation suffer from the heavy bias toward detecting frequent objects in the long-tailed setting. Existing methods address this…
MoViNets: Mobile Video Networks for Efficient Video Recognition
Dan Kondratyuk, Liangzhe Yuan, Yandong Li +4
We present Mobile Video Networks (MoViNets), a family of computation and memory efficient video networks that can operate on streaming video for online inference. 3D convolutional…
MosaicOS: A Simple and Effective Use of Object-Centric Images for Long-Tailed Object Detection
Cheng Zhang, Tai-Yu Pan, Yandong Li +5
Many objects do not appear frequently enough in complex scenes (e.g., certain handbags in living rooms) for training an accurate object detector, but are often found frequently by…
Ranking Neural Checkpoints
Yandong Li, Xuhui Jia, Ruoxin Sang +4
This paper is concerned with ranking many pre-trained deep neural networks (DNNs), called checkpoints, for the transfer learning to a downstream task. Thanks to the broad use of DN…
Improving Object Detection with Selective Self-supervised Self-training
Yandong Li, Di Huang, Danfeng Qin +2
We study how to leverage Web images to augment human-curated object detection datasets. Our approach is two-pronged. On the one hand, we retrieve Web images by image-to-image searc…
Neural Networks Are More Productive Teachers Than Human Raters: Active Mixup for Data-Efficient Knowledge Distillation from a Blackbox Model
Dongdong Wang, Yandong Li, Liqiang Wang +1
We study how to train a student deep neural network for visual recognition by distilling knowledge from a blackbox teacher model in a data-efficient manner. Progress on this proble…
BachGAN: High-Resolution Image Synthesis from Salient Object Layout
Yandong Li, Yu Cheng, Zhe Gan +3
We propose a new task towards more practical application for image generation - high-quality image synthesis from salient object layout. This new setting allows users to provide th…
AdaFilter: Adaptive Filter Fine-tuning for Deep Transfer Learning
Yunhui Guo, Yandong Li, Liqiang Wang +1
There is an increasing number of pre-trained deep neural network models. However, it is still unclear how to effectively use these models for a new task. Transfer learning, which a…
Transferring Robustness for Graph Neural Network Against Poisoning Attacks
Xianfeng Tang, Yandong Li, Yiwei Sun +3
Graph neural networks (GNNs) are widely used in many applications. However, their robustness against adversarial attacks is criticized. Prior studies show that using unnoticeable m…
NATTACK: Learning the Distributions of Adversarial Examples for an Improved Black-Box Attack on Deep Neural Networks
Yandong Li, Lijun Li, Liqiang Wang +2
Powerful adversarial attack methods are vital for understanding how to construct robust deep neural networks (DNNs) and for thoroughly testing defense techniques. In this paper, we…
Joint Modeling of Dense and Incomplete Trajectories for Citywide Traffic Volume Inference
Xianfeng Tang, Boqing Gong, Yanwei Yu +4
Real-time traffic volume inference is key to an intelligent city. It is a challenging task because accurate traffic volumes on the roads can only be measured at certain locations w…
Depthwise Convolution is All You Need for Learning Multiple Visual Domains
Yunhui Guo, Yandong Li, Rogerio Feris +2
There is a growing interest in designing models that can deal with images from different visual domains. If there exists a universal structure in different visual domains that can…
StNet: Local and Global Spatial-Temporal Modeling for Action Recognition
Dongliang He, Zhichao Zhou, Chuang Gan +5
Despite the success of deep learning for static image understanding, it remains unclear what are the most effective network architectures for the spatial-temporal modeling in video…