45 citations · 82 across the 16 of their papers we have counts for
18 papers
ERA: Entropy-Guided Visual Token Pruning with Rectified Attention for Efficient MLLMs
Yuhao Wang, Mu Qiao, Haiwen Diao +5
Multimodal Large Language Models (MLLMs) incur prohibitive inference costs due to long visual token sequences. Training-free visual token reduction provides an efficient solution.…
VISTA-Bench: Do Vision-Language Models Really Understand Visualized Text as Well as Pure Text?
Qing'an Liu, Juntong Feng, Yuhao Wang +6
Vision-Language Models (VLMs) have achieved impressive performance in cross-modal understanding across textual and visual inputs, yet existing benchmarks predominantly focus on pur…
Regularizing Subspace Redundancy of Low-Rank Adaptation
Yue Zhu, Haiwen Diao, Shang Gao +8
Low-Rank Adaptation (LoRA) and its variants have delivered strong capability in Parameter-Efficient Transfer Learning (PETL) by minimizing trainable parameters and benefiting from…
End-to-End Vision Tokenizer Tuning
Wenxuan Wang, Fan Zhang, Yufeng Cui +5
Existing vision tokenization isolates the optimization of vision tokenizers from downstream training, implicitly assuming the visual tokens can generalize well across various tasks…
KARST: Multi-Kernel Kronecker Adaptation with Re-Scaling Transmission for Visual Classification
Yue Zhu, Haiwen Diao, Shang Gao +2
Fine-tuning pre-trained vision models for specific tasks is a common practice in computer vision. However, this process becomes more expensive as models grow larger. Recently, para…
EVEv2: Improved Baselines for Encoder-Free Vision-Language Models
Haiwen Diao, Xiaotong Li, Yufeng Cui +6
Existing encoder-free vision-language models (VLMs) are rapidly narrowing the performance gap with their encoder-based counterparts, highlighting the promising potential for unifie…
MoTrans: Customized Motion Transfer with Text-driven Video Diffusion Models
Xiaomin Li, Xu Jia, Qinghe Wang +5
Existing pretrained text-to-video (T2V) models have demonstrated impressive abilities in generating realistic videos with basic motion or camera movement. However, these models exh…
Autoregressive Video Generation without Vector Quantization
Haoge Deng, Ting Pan, Haiwen Diao +6
This paper presents a novel approach that enables autoregressive video generation with high efficiency. We propose to reformulate the video generation problem as a non-quantized au…
LLMs Can Evolve Continually on Modality for X-Modal Reasoning
Jiazuo Yu, Haomiao Xiong, Lu Zhang +7
Multimodal Large Language Models (MLLMs) have gained significant attention due to their impressive capabilities in multimodal understanding. However, existing methods rely heavily…
GSSF: Generalized Structural Sparse Function for Deep Cross-modal Metric Learning
Haiwen Diao, Ying Zhang, Shang Gao +3
Cross-modal metric learning is a prominent research topic that bridges the semantic heterogeneity between vision and language. Existing methods frequently utilize simple cosine or…
SHERL: Synthesizing High Accuracy and Efficient Memory for Resource-Limited Transfer Learning
Haiwen Diao, Bo Wan, Xu Jia +4
Parameter-efficient transfer learning (PETL) has emerged as a flourishing research field for adapting large pre-trained models to downstream tasks, greatly reducing trainable param…
DenseFusion-1M: Merging Vision Experts for Comprehensive Multimodal Perception
Xiaotong Li, Fan Zhang, Haiwen Diao +3
Existing Multimodal Large Language Models (MLLMs) increasingly emphasize complex understanding of various visual elements, including multiple objects, text information, and spatial…
Unveiling Encoder-Free Vision-Language Models
Haiwen Diao, Yufeng Cui, Xiaotong Li +3
Existing vision-language models (VLMs) mostly rely on vision encoders to extract visual features followed by large language models (LLMs) for visual-language tasks. However, the vi…
Deep Boosting Learning: A Brand-new Cooperative Approach for Image-Text Matching
Haiwen Diao, Ying Zhang, Shang Gao +2
Image-text matching remains a challenging task due to heterogeneous semantic diversity across modalities and insufficient distance separability within triplets. Different from prev…
Exploring Dynamic Transformer for Efficient Object Tracking
Jiawen Zhu, Xin Chen, Haiwen Diao +6
The speed-precision trade-off is a critical problem for visual object tracking which usually requires low latency and deployment on constrained resources. Existing solutions for ef…
UniPT: Universal Parallel Tuning for Transfer Learning with Efficient Parameter and Memory
Haiwen Diao, Bo Wan, Ying Zhang +3
Parameter-efficient transfer learning (PETL), i.e., fine-tuning a small portion of parameters, is an effective strategy for adapting pre-trained models to downstream domains. To fu…
Plug-and-Play Regulators for Image-Text Matching
Haiwen Diao, Ying Zhang, Wei Liu +2
Exploiting fine-grained correspondence and visual-semantic alignments has shown great potential in image-text matching. Generally, recent approaches first employ a cross-modal atte…
Similarity Reasoning and Filtration for Image-Text Matching
Haiwen Diao, Ying Zhang, Lin Ma +1
Image-text matching plays a critical role in bridging the vision and language, and great progress has been made by exploiting the global alignment between image and sentence, or lo…