89 citations · 225 across the 10 of their papers we have counts for
11 papers · 1 filter
LoRA Done RITE: Robust Invariant Transformation Equilibration for LoRA Optimization
Jui-Nan Yen, Si Si, Zhao Meng +5
Low-rank adaption (LoRA) is a widely used parameter-efficient finetuning method for LLM that reduces memory requirements. However, current LoRA optimizers lack transformation invar…
Learnable Fourier Features for Multi-Dimensional Spatial Positional Encoding
Yang Li, Si Si, Gang Li +2
Attentional mechanisms are order-invariant. Positional encoding is a crucial component to allow attention-based deep model architectures such as Transformer to address sequences or…
How much progress have we made in neural network training? A New Evaluation Protocol for Benchmarking Optimizers
Yuanhao Xiong, Xuanqing Liu, Li-Cheng Lan +3
Many optimizers have been proposed for training deep neural networks, and they often have multiple hyperparameters, which make it tricky to benchmark their performance. In this wor…
Multi-Stage Influence Function
Hongge Chen, Si Si, Yang Li +4
Multi-stage training and knowledge transfer, from a large-scale pretraining task to various finetuning tasks, have revolutionized natural language processing and computer vision re…
A Unified Framework for Data Poisoning Attack to Graph-based Semi-supervised Learning
Xuanqing Liu, Si Si, Xiaojin Zhu +2
In this paper, we proposed a general framework for data poisoning attacks to graph-based semi-supervised learning (G-SSL). In this framework, we first unify different tasks, goals,…
Neural SDE: Stabilizing Neural ODE Networks with Stochastic Noise
Xuanqing Liu, Tesi Xiao, Si Si +3
Neural Ordinary Differential Equation (Neural ODE) has been proposed as a continuous approximation to the ResNet architecture. Some commonly used regularization mechanisms in discr…