4 papers
UniAPO: Unified Multimodal Automated Prompt Optimization
Qipeng Zhu, Yanzhe Chen, Huasong Zhong +5
Prompting is fundamental to unlocking the full potential of large language models. To automate and enhance this process, automatic prompt optimization (APO) has been developed, dem…
UniCode: Cascaded Large-scale Codebooks for Unified Multimodal Understanding and Generation
Yanzhe Chen, Huasong Zhong, Yan Li +1
Unified multimodal large language models (MLLMs) have shown promise in jointly advancing multimodal understanding and generation, with visual codebooks discretizing images into tok…
An Embarrassingly Simple Data Selection Strategy for Million-Scale Multi-Modal Instruction-Tuning
Mengyao Lyu, Yan Li, Huasong Zhong +5
Recent research demonstrates that pretrained large language models (LLMs) require minimal but carefully curated data for instruction-tuning. Building on LLMs, multi-modal models fu…
EVLM: An Efficient Vision-Language Model for Visual Understanding
Kaibing Chen, Dong Shen, Hanwen Zhong +14
In the field of multi-modal language models, the majority of methods are built on an architecture similar to LLaVA. These models use a single-layer ViT feature as a visual prompt,…