103 citations · 469 across the 31 of their papers we have counts for
16 papers · 1 filter
POME: Post Optimization Model Edit via Muon-style Projection
Yong Liu, Di Fu, Yang Luo +4
We introduce Post-Optimization Model Edit (POME), a new algorithm that enhances the performance of fine-tuned large language models using only their pretrained and fine-tuned check…
Sparse MeZO: Less Parameters for Better Performance in Zeroth-Order LLM Fine-Tuning
Yong Liu, Zirui Zhu, Chaoyu Gong +3
While fine-tuning large language models (LLMs) for specific tasks often yields impressive results, it comes at the cost of memory inefficiency due to back-propagation in gradient-b…
Revisiting Personalized Federated Learning: Robustness Against Backdoor Attacks
Zeyu Qin, Liuyi Yao, Daoyuan Chen +3
In this work, besides improving prediction accuracy, we study whether personalization could bring robustness benefits to backdoor attacks. We conduct the first study of backdoor at…
Efficient Non-Parametric Optimizer Search for Diverse Tasks
Ruochen Wang, Yuanhao Xiong, Minhao Cheng +1
Efficient and automated design of optimizers plays a crucial role in full-stack AutoML systems. However, prior methods in optimizer search are often limited by their scalability, g…
FedDM: Iterative Distribution Matching for Communication-Efficient Federated Learning
Yuanhao Xiong, Ruochen Wang, Minhao Cheng +2
Federated learning~(FL) has recently attracted increasing attention from academia and industry, with the ultimate goal of achieving collaborative training under privacy and communi…
RANK-NOSH: Efficient Predictor-Based Architecture Search via Non-Uniform Successive Halving
Ruochen Wang, Xiangning Chen, Minhao Cheng +2
Predictor-based algorithms have achieved remarkable performance in the Neural Architecture Search (NAS) tasks. However, these methods suffer from high computation costs, as trainin…
Rethinking Architecture Selection in Differentiable NAS
Ruochen Wang, Minhao Cheng, Xiangning Chen +2
Differentiable Neural Architecture Search is one of the most popular Neural Architecture Search (NAS) methods for its search efficiency and simplicity, accomplished by jointly opti…
Concurrent Adversarial Learning for Large-Batch Training
Yong Liu, Xiangning Chen, Minhao Cheng +2
Large-batch training has become a commonly used technique when training neural networks with a large number of GPU/TPU processors. As batch size increases, stochastic optimizers te…
Self-Progressing Robust Training
Minhao Cheng, Pin-Yu Chen, Sijia Liu +3
Enhancing model robustness under new and even adversarial environments is a crucial milestone toward building trustworthy machine learning systems. Current robust training methods…
Voting based ensemble improves robustness of defensive models
Devvrit, Minhao Cheng, Cho-Jui Hsieh +1
Developing robust models against adversarial perturbations has been an active area of research and many algorithms have been proposed to train individual robust models. Taking thes…
DrNAS: Dirichlet Neural Architecture Search
Xiangning Chen, Ruochen Wang, Minhao Cheng +2
This paper proposes a novel differentiable architecture search method by formulating it into a distribution learning problem. We treat the continuously relaxed architecture mixing…
CAT: Customized Adversarial Training for Improved Robustness
Minhao Cheng, Qi Lei, Pin-Yu Chen +2
Adversarial training has become one of the most effective methods for improving robustness of neural networks. However, it often suffers from poor generalization on both clean and…
Sign-OPT: A Query-Efficient Hard-label Adversarial Attack
Minhao Cheng, Simranjit Singh, Patrick Chen +3
We study the most practical problem setup for evaluating adversarial robustness of a machine learning system with limited access: the hard-label black-box attack setting for genera…
Query-Efficient Hard-label Black-box Attack:An Optimization-based Approach
Minhao Cheng, Thong Le, Pin-Yu Chen +3
We study the problem of attacking a machine learning model in the hard-label black-box setting, where no model information is revealed except that the attacker can make queries to…
Seq2Sick: Evaluating the Robustness of Sequence-to-Sequence Models with Adversarial Examples
Minhao Cheng, Jinfeng Yi, Pin-Yu Chen +2
Crafting adversarial examples has become an important technique to evaluate the robustness of deep neural networks (DNNs). However, most existing works focus on attacking the image…
Towards Robust Neural Networks via Random Self-ensemble
Xuanqing Liu, Minhao Cheng, Huan Zhang +1
Recent studies have revealed the vulnerability of deep neural networks: A small adversarial perturbation that is imperceptible to human can easily make a well-trained deep neural n…