15 papers
CFCML: A Coarse-to-Fine Crossmodal Learning Framework For Disease Diagnosis Using Multimodal Images and Tabular Data
Tianling Liu, Hongying Liu, Fanhua Shang +3
In clinical practice, crossmodal information including medical images and tabular data is essential for disease diagnosis. There exists a significant modality gap between these dat…
PDD: Manifold-Prior Diverse Distillation for Medical Anomaly Detection
Xijun Lu, Hongying Liu, Fanhua Shang +2
Medical image anomaly detection faces unique challenges due to subtle, heterogeneous anomalies embedded in complex anatomical structures. Through systematic Grad-CAM analysis, we r…
SSL-SSAW: Self-Supervised Learning with Sigmoid Self-Attention Weighting for Question-Based Sign Language Translation
Zekang Liu, Wei Feng, Fanhua Shang +3
Sign Language Translation (SLT) bridges the communication gap between deaf people and hearing people, where dialogue provides crucial contextual cues to aid in translation. Buildin…
LightVLM: Acceleraing Large Multimodal Models with Pyramid Token Merging and KV Cache Compression
Lianyu Hu, Fanhua Shang, Wei Feng +1
In this paper, we introduce LightVLM, a simple but effective method that can be seamlessly deployed upon existing Vision-Language Models (VLMs) to greatly accelerate the inference…
FedSWA: Improving Generalization in Federated Learning with Highly Heterogeneous Data via Momentum-Based Stochastic Controlled Weight Averaging
Liu junkang, Yuanyuan Liu, Fanhua Shang +3
For federated learning (FL) algorithms such as FedSAM, their generalization capability is crucial for real-word applications. In this paper, we revisit the generalization problem i…
Beyond Background Shift: Rethinking Instance Replay in Continual Semantic Segmentation
Hongmei Yin, Tingliang Feng, Fan Lyu +4
In this work, we focus on continual semantic segmentation (CSS), where segmentation networks are required to continuously learn new classes without erasing knowledge of previously…
iLLaVA: An Image is Worth Fewer Than 1/3 Input Tokens in Large Multimodal Models
Lianyu Hu, Liqing Gao, Fanhua Shang +2
Recent methods have made notable progress in accelerating Large Vision-Language Models (LVLMs) by exploiting the inherent redundancy in visual inputs. Most existing approaches, how…
Deep Correlated Prompting for Visual Recognition with Missing Modalities
Lianyu Hu, Tongkai Shi, Wei Feng +2
Large-scale multimodal models have shown excellent performance over a series of tasks powered by the large corpus of paired multimodal training data. Generally, they are always ass…
Pose-Guided Fine-Grained Sign Language Video Generation
Tongkai Shi, Lianyu Hu, Fanhua Shang +3
Sign language videos are an important medium for spreading and learning sign language. However, most existing human image synthesis methods produce sign language images with detail…
Towards stable training of parallel continual learning
Li Yuepan, Fan Lyu, Yuyang Li +3
Parallel Continual Learning (PCL) tasks investigate the training methods for continual learning with multi-source input, where data from different tasks are learned as they arrive.…
Completed Feature Disentanglement Learning for Multimodal MRIs Analysis
Tianling Liu, Hongying Liu, Fanhua Shang +3
Multimodal MRIs play a crucial role in clinical diagnosis and treatment. Feature disentanglement (FD)-based methods, aiming at learning superior feature representations for multimo…
Controllable Continual Test-Time Adaptation
Ziqi Shi, Fan Lyu, Ye Liu +5
Continual Test-Time Adaptation (CTTA) is an emerging and challenging task where a model trained in a source domain must adapt to continuously changing conditions during testing, wi…
Overcoming Domain Drift in Online Continual Learning
Fan Lyu, Daofeng Liu, Linglan Zhao +5
Online Continual Learning (OCL) empowers machine learning models to acquire new knowledge online across a sequence of tasks. However, OCL faces a significant challenge: catastrophi…
Elastic Multi-Gradient Descent for Parallel Continual Learning
Fan Lyu, Wei Feng, Yuepan Li +4
The goal of Continual Learning (CL) is to continuously learn from new data streams and accomplish the corresponding tasks. Previously studied CL assumes that data are given in sequ…
Long-Tailed Learning as Multi-Objective Optimization
Weiqi Li, Fan Lyu, Fanhua Shang +2
Real-world data is extremely imbalanced and presents a long-tailed distribution, resulting in models that are biased towards classes with sufficient samples and perform poorly on r…