22 citations · 48 across the 13 of their papers we have counts for
13 papers
Aurora: A Leverage-Aware Spectral Optimizer
Alec Dewulf, Dhruv Pai, Li Yang +2
We show that for tall matrix parameters, like projection matrices in the MLP layers, the Muon update can have row norms that are arbitrarily non-uniform. This can lead to a self-re…
Dominant-Layer ZO: A Single Layer Dominates Zeroth-Order Fine-Tuning of LLMs
Wanhao Yu, Ziyan Wang, Zheng Wang +7
Zeroth-order (ZO) optimization enables memory-efficient fine-tuning of large language models (LLMs) using only forward passes, but it remains unclear how useful adaptation is distr…
LP-3DGS: Learning to Prune 3D Gaussian Splatting
Zhaoliang Zhang, Tianchen Song, Yongjae Lee +4
Recently, 3D Gaussian Splatting (3DGS) has become one of the mainstream methodologies for novel view synthesis (NVS) due to its high quality and fast rendering speed. However, as a…
MF-NeRF: Memory Efficient NeRF with Mixed-Feature Hash Table
Yongjae Lee, Li Yang, Deliang Fan
Neural radiance field (NeRF) has shown remarkable performance in generating photo-realistic novel views. Among recent NeRF related research, the approaches that involve the utiliza…
Model Extraction Attacks on Split Federated Learning
Jingtao Li, Adnan Siraj Rakin, Xing Chen +4
Federated Learning (FL) is a popular collaborative learning scheme involving multiple clients and a server. FL focuses on protecting clients' data but turns out to be highly vulner…
Efficient Self-supervised Continual Learning with Progressive Task-correlated Layer Freezing
Li Yang, Sen Lin, Fan Zhang +2
Inspired by the success of Self-supervised learning (SSL) in learning visual representations from unlabeled data, a few recent works have studied SSL in the context of continual le…
Beyond Not-Forgetting: Continual Learning with Backward Knowledge Transfer
Sen Lin, Li Yang, Deliang Fan +1
By learning a sequence of tasks continually, an agent in continual learning (CL) can improve the learning performance of both a new task and `old' tasks by leveraging the forward k…
TRGP: Trust Region Gradient Projection for Continual Learning
Sen Lin, Li Yang, Deliang Fan +1
Catastrophic forgetting is one of the major challenges in continual learning. To address this issue, some existing methods put restrictive constraints on the optimization space of…
Gradient-based Novelty Detection Boosted by Self-supervised Binary Classification
Jingbo Sun, Li Yang, Jiaxin Zhang +4
Novelty detection aims to automatically identify out-of-distribution (OOD) data, without any prior knowledge of them. It is a critical step in data monitoring, behavior analysis an…
GROWN: GRow Only When Necessary for Continual Learning
Li Yang, Sen Lin, Junshan Zhang +1
Catastrophic forgetting is a notorious issue in deep learning, referring to the fact that Deep Neural Networks (DNN) could forget the knowledge about earlier tasks when learning ne…
RA-BNN: Constructing Robust & Accurate Binary Neural Network to Simultaneously Defend Adversarial Bit-Flip Attack and Improve Accuracy
Adnan Siraj Rakin, Li Yang, Jingtao Li +5
Recently developed adversarial weight attack, a.k.a. bit-flip attack (BFA), has shown enormous success in compromising Deep Neural Network (DNN) performance with an extremely small…
:Dynamic Additive Attention Adaption for Memory-EfficientOn-Device Multi-Domain Learning
Li Yang, Adnan Siraj Rakin, Deliang Fan
Nowadays, one practical limitation of deep neural network (DNN) is its high degree of specialization to a single task or domain (e.g., one visual domain). It motivates researchers…
MetaGater: Fast Learning of Conditional Channel Gated Networks via Federated Meta-Learning
Sen Lin, Li Yang, Zhezhi He +2
While deep learning has achieved phenomenal successes in many AI applications, its enormous model size and intensive computation requirements pose a formidable challenge to the dep…