13 citations · 29 across the 17 of their papers we have counts for
17 papers
CARE-LoRA: Compressed Activation REconstruction for Memory-Efficient LoRA
Gengyu Zhang, Haiyin Ran, Zhengbao He +4
As the scale of large pre-trained models continues to grow, fine-tuning them under limited memory budgets has become increasingly challenging. Low-Rank Adaptation (LoRA), currently…
SAGE: Retain-Aware Post-Hoc Sanitization of Final Unlearning Vector
Jingyuan Zhang, Yucheng Bai, Peixi Wen +6
Large Language Model (LLM) unlearning aims to remove undesirable knowledge or behaviors while preserving retained capabilities. Current unlearning methods all involve a trade-off b…
Compress then Merge: From Multiple LoRAs into One Low-Rank Adapter
Zhengbao He, Ruiqi Ding, Zhehao Huang +3
Low-rank adaptation (LoRA) enables parameter-efficient specialization of foundation models, but the proliferation of task-specific adapters fragments capabilities across many adapt…
RAIN-Merging: A Gradient-Free Method to Enhance Instruction Following in Large Reasoning Models with Preserved Thinking Format
Zhehao Huang, Yuhang Liu, Baijiong Lin +5
Large reasoning models (LRMs) excel at a long chain of reasoning but often fail to faithfully follow instructions regarding output format, constraints, or specific requirements. We…
T2I-ConBench: Text-to-Image Benchmark for Continual Post-training
Zhehao Huang, Yuhang Liu, Yixin Lou +7
Continual post-training adapts a single text-to-image diffusion model to learn new tasks without incurring the cost of separate models, but naive post-training causes forgetting of…
A Unified Gradient-based Framework for Task-agnostic Continual Learning-Unlearning
Zhehao Huang, Xinwen Cheng, Jie Zhang +5
Recent advancements in deep models have highlighted the need for intelligent systems that combine continual learning (CL) for knowledge acquisition with machine unlearning (MU) for…
MUSO: Achieving Exact Machine Unlearning in Over-Parameterized Regimes
Ruikai Yang, Mingzhen He, Zhengbao He +2
Machine unlearning (MU) is to make a well-trained model behave as if it had never been trained on specific data. In today's over-parameterized models, dominated by neural networks,…
Unified Gradient-Based Machine Unlearning with Remain Geometry Enhancement
Zhehao Huang, Xinwen Cheng, JingHao Zheng +4
Machine unlearning (MU) has emerged to enhance the privacy and trustworthiness of deep neural networks. Approximate MU is a practical method for large-scale models. Our investigati…
Flat-LoRA: Low-Rank Adaptation over a Flat Loss Landscape
Tao Li, Zhengbao He, Yujun Li +3
Fine-tuning large-scale pre-trained models is prohibitively expensive in terms of computation and memory costs. Low-Rank Adaptation (LoRA), a popular Parameter-Efficient Fine-Tunin…
Pursuing Feature Separation based on Neural Collapse for Out-of-Distribution Detection
Yingwen Wu, Ruiji Yu, Xinwen Cheng +2
In the open world, detecting out-of-distribution (OOD) data, whose labels are disjoint with those of in-distribution (ID) samples, is important for reliable deep neural networks (D…
Towards Natural Machine Unlearning
Zhengbao He, Tao Li, Xinwen Cheng +2
Machine unlearning (MU) aims to eliminate information that has been learned from specific training data, namely forgetting data, from a pre-trained model. Currently, the mainstream…
Friendly Sharpness-Aware Minimization
Tao Li, Pan Zhou, Zhengbao He +2
Sharpness-Aware Minimization (SAM) has been instrumental in improving deep neural network training by minimizing both training loss and loss sharpness. Despite the practical succes…
Remaining-data-free Machine Unlearning by Suppressing Sample Contribution
Xinwen Cheng, Zhehao Huang, Wenxin Zhou +4
Machine unlearning (MU) aims to remove the influence of specific training samples from a well-trained model, a task of growing importance due to the ``right to be forgotten.'' The…
Investigating Catastrophic Overfitting in Fast Adversarial Training: A Self-fitting Perspective
Zhengbao He, Tao Li, Sizhe Chen +1
Although fast adversarial training provides an efficient approach for building robust networks, it may suffer from a serious problem known as catastrophic overfitting (CO), where m…
Trainable Weight Averaging: Accelerating Training and Improving Generalization
Tao Li, Zhehao Huang, Yingwen Wu +4
Weight averaging is a widely used technique for accelerating training and improving the generalization of deep neural networks (DNNs). While existing approaches like stochastic wei…
Universal Adversarial Attack on Attention and the Resulting Dataset DAmageNet
Sizhe Chen, Zhengbao He, Chengjin Sun +2
Adversarial attacks on deep neural networks (DNNs) have been found for several years. However, the existing adversarial attacks have high success rates only when the information of…
DAmageNet: A Universal Adversarial Dataset
Sizhe Chen, Xiaolin Huang, Zhengbao He +1
It is now well known that deep neural networks (DNNs) are vulnerable to adversarial attack. Adversarial samples are similar to the clean ones, but are able to cheat the attacked DN…