7 papers
Where Is the "Self" of a Large Language Model? -- From Self-Index to Self-Reference
Yuchen Li, Zongqi Fan, Xintong Li +2
We separate two functional levels of self familiar from biology: a self-index, which separates one's own from external material and shapes processing, as corollary discharge does i…
Rethinking the Information Bottleneck: Structured Decomposition under Label-Induced Partitions
Jingyao Zhang, Yuxuan Li, Lu Han +2
Standard information bottleneck (IB) regularization constrains representations via a single scalar I(Z;X), implicitlytreating all information as homogeneous. However, a single glob…
Backpropagated Output Momentum: Relocating Optimizer History from Parameters to Task Space
Yuchen Li, Zongqi Fan, Nguyen H. Tran +1
Optimizer momentum is usually stored as a parameter-sized moving average of past gradients, which makes history costly and fixes each past signal in the coordinates in which it was…
Routing in Gradient Space: Balanced Usage Is Not Expert Specialization
Yuchen Li, Mingyu Du, Zongqi Fan +2
Sparse expert models can distribute traffic evenly while still grouping incompatible training signals within the same experts. We study routing as a gradient-partitioning problem a…
Generative Modeling in Protein Design: Neural Representations, Conditional Generation, and Evaluation Standards
Senura Hansaja Wanasekara, Minh-Duong Nguyen, Xiaochen Liu +2
Generative modeling has become a central paradigm in protein research, extending machine learning beyond structure prediction toward sequence design, backbone generation, inverse f…
Computation and Communication Efficient Federated Unlearning via On-server Gradient Conflict Mitigation and Expression
Minh-Duong Nguyen, Senura Hansaja, Le-Tuan Nguyen +4
Federated Unlearning (FUL) aims to remove specific participants' data contributions from a trained Federated Learning model, thereby ensuring data privacy and compliance with regul…
Knowledge Abstraction for Knowledge-based Semantic Communication: A Generative Causality Invariant Approach
Minh-Duong Nguyen, Quoc-Viet Pham, Nguyen H. Tran +3
In this study, we design a low-complexity and generalized AI model that can capture common knowledge to improve data reconstruction of the channel decoder for semantic communicatio…