244 citations · 389 across the 28 of their papers we have counts for
17 papers · 1 filter
How Well Do Models Follow Visual Instructions? VIBE: A Systematic Benchmark for Visual Instruction-Driven Image Editing
Huanyu Zhang, Xuehai Bai, Chengzu Li +9
Recent generative models have achieved remarkable progress in image editing. However, existing systems and benchmarks remain largely text-guided. In contrast, human communication i…
SimScale: Learning to Drive via Real-World Simulation at Scale
Haochen Tian, Tianyu Li, Haochen Liu +11
Achieving fully autonomous driving systems requires learning rational decisions in a wide span of scenarios, including safety-critical and out-of-distribution ones. However, such c…
Latent Sketchpad: Sketching Visual Thoughts to Elicit Multimodal Reasoning in MLLMs
Huanyu Zhang, Wenshan Wu, Chengzu Li +9
While Multimodal Large Language Models (MLLMs) excel at visual understanding, they often struggle in complex scenarios that require visual planning and imagination. Inspired by how…
MME-Unify: A Comprehensive Benchmark for Unified Multimodal Understanding and Generation Models
Wulin Xie, Yi-Fan Zhang, Chaoyou Fu +6
Existing MLLM benchmarks face significant challenges in evaluating Unified MLLMs (U-MLLMs) due to: 1) lack of standardized benchmarks for traditional tasks, leading to inconsistent…
Aligning Multimodal LLM with Human Preference: A Survey
Tao Yu, Yi-Fan Zhang, Chaoyou Fu +14
Large language models (LLMs) can handle a wide variety of general tasks with simple prompts, without the need for task-specific training. Multimodal Large Language Models (MLLMs),…
MME-RealWorld: Could Your Multimodal LLM Challenge High-Resolution Real-World Scenarios that are Difficult for Humans?
Yi-Fan Zhang, Huanyu Zhang, Haochen Tian +10
Comprehensive evaluation of Multimodal Large Language Models (MLLMs) has recently garnered widespread attention in the research community. However, we observe that existing benchma…
Debiasing Multimodal Large Language Models via Penalization of Language Priors
YiFan Zhang, Yang Shi, Weichen Yu +6
In the realms of computer vision and natural language processing, Multimodal Large Language Models (MLLMs) have become indispensable tools, proficient in generating textual respons…
Logical Closed Loop: Uncovering Object Hallucinations in Large Vision-Language Models
Junfei Wu, Qiang Liu, Ding Wang +4
Object hallucination has been an Achilles' heel which hinders the broader applications of large vision-language models (LVLMs). Object hallucination refers to the phenomenon that t…
AE-NeRF: Audio Enhanced Neural Radiance Field for Few Shot Talking Head Synthesis
Dongze Li, Kang Zhao, Wei Wang +4
Audio-driven talking head synthesis is a promising topic with wide applications in digital human, film making and virtual reality. Recent NeRF-based approaches have shown superiori…
End-to-end Alternating Optimization for Real-World Blind Super Resolution
Zhengxiong Luo, Yan Huang, Shang Li +2
Blind Super-Resolution (SR) usually involves two sub-problems: 1) estimating the degradation of the given low-resolution (LR) image; 2) super-resolving the LR image to its high-res…
GaFET: Learning Geometry-aware Facial Expression Translation from In-The-Wild Images
Tianxiang Ma, Bingchuan Li, Qian He +2
While current face animation methods can manipulate expressions individually, they suffer from several limitations. The expressions manipulated by some motion-based facial reenactm…
Realistic Unsupervised CLIP Fine-tuning with Universal Entropy Optimization
Jian Liang, Lijun Sheng, Zhengbo Wang +2
The emergence of vision-language models, such as CLIP, has spurred a significant research effort towards their application for downstream supervised learning tasks. Although some p…
Improving Zero-Shot Generalization for CLIP with Synthesized Prompts
Zhengbo Wang, Jian Liang, Ran He +3
With the growing interest in pretrained vision-language models like CLIP, recent research has focused on adapting these models to downstream tasks. Despite achieving promising resu…
Semantic Prompt for Few-Shot Image Recognition
Wentao Chen, Chenyang Si, Zhang Zhang +3
Few-shot learning is a challenging problem since only a few examples are provided to recognize a new class. Several recent studies exploit additional semantic information, e.g. tex…
Exploiting Semantic Attributes for Transductive Zero-Shot Learning
Zhengbo Wang, Jian Liang, Zilei Wang +1
Zero-shot learning (ZSL) aims to recognize unseen classes by generalizing the relation between visual features and semantic attributes learned from the seen classes. A recent parad…
VideoFusion: Decomposed Diffusion Models for High-Quality Video Generation
Zhengxiong Luo, Dayou Chen, Yingya Zhang +6
A diffusion probabilistic model (DPM), which constructs a forward diffusion process by gradually adding noise to data points and learns the reverse denoising process to generate ne…
DeltaEdit: Exploring Text-free Training for Text-Driven Image Manipulation
Yueming Lyu, Tianwei Lin, Fu Li +3
Text-driven image manipulation remains challenging in training or inference flexibility. Conditional generative models depend heavily on expensive annotated training data. Meanwhil…