7 papers · 1 filter
Spectral Gating Networks
Jusheng Zhang, Yijia Fan, Kaitong Cai +5
Gating mechanisms are ubiquitous, yet a complementary question in feed-forward networks remains under-explored: how to introduce frequency-rich expressivity without sacrificing sta…
Rational ANOVA Networks
Jusheng Zhang, Ningyuan Liu, Qinhan Lyu +2
Deep neural networks typically treat nonlinearities as fixed primitives (e.g., ReLU), limiting both interpretability and the granularity of control over the induced function class.…
RevFFN: Memory-Efficient Full-Parameter Fine-Tuning of Mixture-of-Experts LLMs with Reversible Blocks
Ningyuan Liu, Jing Yang, Kaitong Cai +1
Full parameter fine tuning is a key technique for adapting large language models (LLMs) to downstream tasks, but it incurs substantial memory overhead due to the need to cache exte…
Causal Invariance and Counterfactual Learning Driven Cooperative Game for Multi-Label Classification
Yijia Fan, Jusheng Zhang, Kaitong Cai +2
Multi-label classification (MLC) remains vulnerable to label imbalance, spurious correlations, and distribution shifts, challenges that are particularly detrimental to rare label p…
Guardian: Decoupling Exploration from Safety in Reinforcement Learning
Kaitong Cai, Jusheng Zhang, Jing Yang +1
Hybrid offline--online reinforcement learning (O2O RL) promises both sample efficiency and robust exploration, but suffers from instability due to distribution shift between offlin…
Backward-Friendly Optimization: Training Large Language Models with Approximate Gradients under Memory Constraints
Jing Yang, Kaitong Cai, Yijia Fan +2
Full fine-tuning of Large Language Models (LLMs) is notoriously memory-intensive, primarily because conventional optimizers such as SGD or Adam assume access to exact gradients der…