1.5k citations · 3.6k across the 48 of their papers we have counts for
6 papers · 1 filter
Visual Instruction Tuning with Chain of Region-of-Interest
Yixin Chen, Shuai Zhang, Boran Han +1
High-resolution (HR) images are pivotal for enhancing the recognition and understanding capabilities of multimodal large language models (MLLMs). However, directly increasing image…
Multimodal Instruction Tuning with Hybrid State Space Models
Jianing Zhou, Han Li, Shuai Zhang +5
Handling lengthy context is crucial for enhancing the recognition and understanding capabilities of multimodal large language models (MLLMs) in applications such as processing high…
Bridging Remote Sensors with Multisensor Geospatial Foundation Models
Boran Han, Shuai Zhang, Xingjian Shi +1
In the realm of geospatial analysis, the diversity of remote sensors, encompassing both optical and microwave technologies, offers a wealth of distinct observational capabilities.…
CaMML: Context-Aware Multimodal Learner for Large Models
Yixin Chen, Shuai Zhang, Boran Han +2
In this work, we introduce Context-Aware MultiModal Learner (CaMML), for tuning large multimodal models (LMMs). CaMML, a lightweight module, is crafted to seamlessly integrate mult…
Lightweight In-Context Tuning for Multimodal Unified Models
Yixin Chen, Shuai Zhang, Boran Han +1
In-context learning (ICL) involves reasoning from given contextual examples. As more modalities comes, this procedure is becoming more challenging as the interleaved input modaliti…
Prompt Pre-Training with Twenty-Thousand Classes for Open-Vocabulary Visual Recognition
Shuhuai Ren, Aston Zhang, Yi Zhu +5
This work proposes POMP, a prompt pre-training method for vision-language models. Being memory and computation efficient, POMP enables the learned prompt to condense semantic infor…