1 citations · 2 across the 25 of their papers we have counts for
28 papers
HyperCLIP++: Fine-tuning CLIP forOpen-vocabulary Semantic Segmentation in Hyperbolic Space
Zelin Peng, Zhengqin Xu, Changsong Wen +4
CLIP, a foundational vision-language model, has emerged as a powerful tool for open-vocabulary semantic segmentation. While freezing CLIP's text encoder is known to preserve its ge…
VaRS-Doc: Interpretation-Aware Variant Representations via Latent Self-Probing for Visual Document Retrieval
Haocheng Wang, Tongkun Guan, Wei Shen +1
Visual document retrieval has recently become increasingly important in applications such as enterprise search, scientific literature discovery, and retrieval-augmented generation.…
AdaThinking-E: One-Token Entropy Regulation for Adaptive Thinking
Zining Wang, Tongkun Guan, Boming Chen +8
Multimodal large language models have demonstrated strong document reasoning capabilities by incorporating explicit thinking processes. While this capability significantly improves…
LightSTAR: Efficient Visual Document Retrieval via Lightweight Selection with Vision-Adaptive Refinement
Tongkun Guan, Haocheng Wang, Wei Shen +1
Visual document retrieval requires rapidly locating relevant pages from large multi-modal corpora in response to user queries. While recent methods powered by Multi-modal Large Lan…
CodePercept: Code-Grounded Visual STEM Perception for MLLMs
Tongkun Guan, Zhibo Yang, Jianqiang Wan +10
When MLLMs fail at Science, Technology, Engineering, and Mathematics (STEM) visual reasoning, a fundamental question arises: is it due to perceptual deficiencies or reasoning limit…
HyperET: Efficient Training in Hyperbolic Space for Multi-modal Large Language Models
Zelin Peng, Zhengqin Xu, Qingyang Liu +2
Multi-modal large language models (MLLMs) have emerged as a transformative approach for aligning visual and textual understanding. They typically require extremely high computation…
Unlocking 3D Affordance Segmentation with 2D Semantic Knowledge
Yu Huang, Zelin Peng, Changsong Wen +2
Affordance segmentation aims to decompose 3D objects into parts that serve distinct functional roles, enabling models to reason about object interactions rather than mere recogniti…
NEARL: Interacted Query Adaptation with Orthogonal Regularization for Medical Vision-Language Understanding
Zelin Peng, Yichen Zhao, Yu Huang +5
Computer-aided medical image analysis is crucial for disease diagnosis and treatment planning. While vision-language models (VLMs) such as CLIP exhibit strong generalization abilit…
MedSeg-R: Reasoning Segmentation in Medical Images with Multimodal Large Language Models
Yu Huang, Zelin Peng, Yichen Zhao +3
Medical image segmentation is crucial for clinical diagnosis, yet existing models are limited by their reliance on explicit human instructions and lack the active reasoning capabil…
Weight Spectra Induced Efficient Model Adaptation
Chongjie Si, Xuankun Yang, Muqing Liu +5
Large-scale foundation models have demonstrated remarkable versatility across a wide range of downstream tasks. However, fully fine-tuning these models incurs prohibitive computati…
MAP: Revisiting Weight Decomposition for Low-Rank Adaptation
Chongjie Si, Zhiyi Shi, Yadao Wang +3
The rapid development of large language models has revolutionized natural language processing, but their fine-tuning remains computationally expensive, hindering broad deployment.…
Revisiting Sparsity Constraint Under High-Rank Property in Partial Multi-Label Learning
Chongjie Si, Yidan Cui, Fuchao Yang +2
Partial Multi-Label Learning (PML) extends the multi-label learning paradigm to scenarios where each sample is associated with a candidate label set containing both ground-truth la…
Why Can Accurate Models Be Learned from Inaccurate Annotations?
Chongjie Si, Yidan Cui, Fuchao Yang +2
Learning from inaccurate annotations has gained significant attention due to the high cost of precise labeling. However, despite the presence of erroneous labels, models trained on…
NAN: A Training-Free Solution to Coefficient Estimation in Model Merging
Chongjie Si, Kangtao Lv, Jingjing Jiang +6
Model merging offers a training-free alternative to multi-task learning by combining independently fine-tuned models into a unified one without access to raw data. However, existin…
Generalized Tensor-based Parameter-Efficient Fine-Tuning via Lie Group Transformations
Chongjie Si, Zhiyi Shi, Xuehui Wang +3
Adapting pre-trained foundation models for diverse downstream tasks is a core practice in artificial intelligence. However, the wide range of tasks and high computational costs mak…
Marten: Visual Question Answering with Mask Generation for Multi-modal Document Understanding
Zining Wang, Tongkun Guan, Pei Fu +7
Multi-modal Large Language Models (MLLMs) have introduced a novel dimension to document understanding, i.e., they endow large language models with visual comprehension capabilities…
A Token-level Text Image Foundation Model for Document Understanding
Tongkun Guan, Zining Wang, Pei Fu +9
In recent years, general visual foundation models (VFMs) have witnessed increasing adoption, particularly as image encoders for popular multi-modal large language models (MLLMs). H…
Dereflection Any Image with Diffusion Priors and Diversified Data
Jichen Hu, Chen Yang, Zanwei Zhou +4
Reflection removal of a single image remains a highly challenging task due to the complex entanglement between target scenes and unwanted reflections. Despite significant progress,…
Realistic Surgical Simulation from Monocular Videos
Kailing Wang, Chen Yang, Keyang Zhao +2
This paper tackles the challenge of automatically performing realistic surgical simulations from readily available surgical videos. Recent efforts have successfully integrated phys…
Task-Specific Directions: Definition, Exploration, and Utilization in Parameter Efficient Fine-Tuning
Chongjie Si, Zhiyi Shi, Shifan Zhang +3
Large language models demonstrate impressive performance on downstream tasks, yet they require extensive resource consumption when fully fine-tuning all parameters. To mitigate thi…
PosFormer: Recognizing Complex Handwritten Mathematical Expression with Position Forest Transformer
Tongkun Guan, Chengyu Lin, Wei Shen +1
Handwritten Mathematical Expression Recognition (HMER) has wide applications in human-machine interaction scenarios, such as digitized education and automated offices. Recently, se…
See Further for Parameter Efficient Fine-tuning by Standing on the Shoulders of Decomposition
Chongjie Si, Xiaokang Yang, Wei Shen
The rapid expansion of large foundation models within the pre-training and fine-tuning framework has underscored that larger models often yield better results. However, the scaling…
Maintaining Structural Integrity in Parameter Spaces for Parameter Efficient Fine-tuning
Chongjie Si, Xuehui Wang, Xue Yang +6
Adapting pre-trained foundation models for various downstream tasks has been prevalent in artificial intelligence. Due to the vast number of tasks and high costs, adjusting all par…
Tendency-driven Mutual Exclusivity for Weakly Supervised Incremental Semantic Segmentation
Chongjie Si, Xuehui Wang, Xiaokang Yang +1
Weakly Incremental Learning for Semantic Segmentation (WILSS) leverages a pre-trained segmentation model to segment new classes using cost-effective and readily available image-lev…