8 papers
SAMPLe: SAM-based Optimizer for Prompt Learning in VLMs
Hossein Rajoli, Fatemeh Lotfi, Niloufar Alipour Talemi +3
Pre-trained Vision-Language Models (VLMs) like CLIP have proven highly effective as foundation models for various downstream applications. However, prompt learning in VLMs encounte…
ZO-SAM: Zero-Order Sharpness-Aware Minimization for Efficient Sparse Training
Jie Ji, Gen Li, Kaiyuan Deng +2
Deep learning models, despite their impressive achievements, suffer from high computational costs and memory requirements, limiting their usability in resource-constrained environm…
Modality-Aware SAM: Sharpness-Aware-Minimization Driven Gradient Modulation for Harmonized Multimodal Learning
Hossein R. Nowdeh, Jie Ji, Xiaolong Ma +1
In multimodal learning, dominant modalities often overshadow others, limiting generalization. We propose Modality-Aware Sharpness-Aware Minimization (M-SAM), a model-agnostic frame…
FairSAM: Fair Classification on Corrupted Image Data Through Sharpness-Aware Minimization
Yucong Dai, Jie Ji, Xiaolong Ma +1
Image classification models trained on clean data often degrade sharply when exposed to corrupted test or deployment data, such as images with impulse noise, Gaussian noise, or env…
Condense, Don't Just Prune: Enhancing Efficiency and Performance in MoE Layer Pruning
Mingyu Cao, Gen Li, Jie Ji +6
Mixture-of-Experts (MoE) has garnered significant attention for its ability to scale up neural networks while utilizing the same or even fewer active parameters. However, MoE does…
Data Overfitting for On-Device Super-Resolution with Dynamic Algorithm and Compiler Co-Design
Gen Li, Zhihao Shu, Jie Ji +4
Deep neural networks (DNNs) are frequently employed in a variety of computer vision applications. Nowadays, an emerging trend in the current video distribution system is to take ad…
FlameFinder: Illuminating Obscured Fire through Smoke with Attentive Deep Metric Learning
Hossein Rajoli, Sahand Khoshdel, Fatemeh Afghah +1
FlameFinder is a deep metric learning (DML) framework designed to accurately detect flames, even when obscured by smoke, using thermal images from firefighter drones during wildfir…
Integrating Fairness and Model Pruning Through Bi-level Optimization
Yucong Dai, Gen Li, Feng Luo +2
Deep neural networks have achieved exceptional results across a range of applications. As the demand for efficient and sparse deep learning models escalates, the significance of mo…