8 papers
Retracing Hodgkin and Huxley: State Recovery Does Not Certify Mechanism
Peiyu Zang, Jiayi Hao, Yongqiang Cai
Predicting observed dynamics does not establish recovery of the underlying physical mechanism. Can machine learning retrace the hidden-state reasoning behind the Hodgkin-Huxley (HH…
FlashPDE: A Drop-In Fused Triton Operator Library for Neural PDE Solvers
Peiyu Zang, Bosen Xie, Ruoxiang Xu +1
Physics-Informed Neural Networks (PINNs) solve PDEs by incorporating physical constraints into neural-network training, but large-scale problems are limited by automatic-differenti…
Vocabulary In-Context Learning in Transformers: Benefits of Positional Encoding
Qian Ma, Ruoxiang Xu, Yongqiang Cai
Numerous studies have demonstrated that the Transformer architecture possesses the capability for in-context learning (ICL). In scenarios involving function approximation, context…
Achieving Universal Approximation and Universal Interpolation via Nonlinearity of Control Families
Yongqiang Cai, Yifei Duan
A significant connection exists between the controllability of dynamical systems and the approximation capabilities of neural networks, where residual networks and vanilla feedforw…
Characterization of phospholipid-cholesterol bilayers as self-assembled amphiphile block polymers that contain headgroups
Xiaoyuan Wang, Fredric S. Cohen, Shixin Xu +1
Cholesterol is known to modulate the structure and function of biological membranes. In this study, we use self-consistent field theory (SCFT) to investigate phospholipid/cholester…
Retrieval Backward Attention without Additional Training: Enhance Embeddings of Large Language Models via Repetition
Yifei Duan, Raphael Shang, Deng Liang +1
Language models can be viewed as functions that embed text into Euclidean space, where the quality of the embedding vectors directly determines model performance, training such neu…
Neural Networks Trained by Weight Permutation are Universal Approximators
Yongqiang Cai, Gaohang Chen, Zhonghua Qiao
The universal approximation property is fundamental to the success of neural networks, and has traditionally been achieved by training networks without any constraints on their par…
A Minimal Control Family of Dynamical Systems for Universal Approximation
Yifei Duan, Yongqiang Cai
The universal approximation property (UAP) holds a fundamental position in deep learning, as it provides a theoretical foundation for the expressive power of neural networks. It is…