7 papers
PPOM: Marginalizing Patch-Grid Phase for CLIP-Based Generalizable Vision-Language Prompt Tuning
Liang Wang, Haoyang Li, Chao Wang +3
Prompt tuning adapts CLIP-based vision-language models with few trainable parameters, yet its predictions remain sensitive to the spatial sampling imposed by a frozen vision transf…
Raw Data Matters: Enhancing Prompt Tuning by Internal Augmentation on Vision-Language Models
Haoyang Li, Liang Wang, Chao Wang +4
For CLIP-based prompt tuning, introducing more data as additional knowledge for enhancing fine-tuning process is proved to be an effective approach. Existing data amplification str…
DPC: Dual-Prompt Collaboration for Tuning Vision-Language Models
Haoyang Li, Liang Wang, Chao Wang +3
The Base-New Trade-off (BNT) problem universally exists during the optimization of CLIP-based prompt tuning, where continuous fine-tuning on base (target) classes leads to a simult…
PhiP-G: Physics-Guided Text-to-3D Compositional Scene Generation
Qixuan Li, Chao Wang, Zongjin He +1
Text-to-3D asset generation has achieved significant optimization under the supervision of 2D diffusion priors. However, when dealing with compositional scenes, existing methods en…
VIKSER: Visual Knowledge-Driven Self-Reinforcing Reasoning Framework
Chao Wang, Chunbai Zhang, Yongxiao Tian +2
Visual reasoning refers to the task of solving questions about visual information. Current visual reasoning methods typically employ pre-trained vision-language model (VLM) strateg…
Reducing Hallucinations: Enhancing VQA for Flood Disaster Damage Assessment with Visual Contexts
Yimin Sun, Chao Wang, Yan Peng
The zero-shot performance of visual question answering (VQA) models relies heavily on prompts. For example, a zero-shot VQA for disaster scenarios could leverage well-designed Chai…
Unleashing the Potential of Large Language Model: Zero-shot VQA for Flood Disaster Scenario
Yimin Sun, Chao Wang, Yan Peng
Visual question answering (VQA) is a fundamental and essential AI task, and VQA-based disaster scenario understanding is a hot research topic. For instance, we can ask questions ab…