7 papers
FlashPDE: A Drop-In Fused Triton Operator Library for Neural PDE Solvers
Peiyu Zang, Bosen Xie, Ruoxiang Xu +1
Physics-Informed Neural Networks (PINNs) solve PDEs by incorporating physical constraints into neural-network training, but large-scale problems are limited by automatic-differenti…
Vocabulary In-Context Learning in Transformers: Benefits of Positional Encoding
Qian Ma, Ruoxiang Xu, Yongqiang Cai
Numerous studies have demonstrated that the Transformer architecture possesses the capability for in-context learning (ICL). In scenarios involving function approximation, context…
Achieving Universal Approximation and Universal Interpolation via Nonlinearity of Control Families
Yongqiang Cai, Yifei Duan
A significant connection exists between the controllability of dynamical systems and the approximation capabilities of neural networks, where residual networks and vanilla feedforw…
Characterization of phospholipid-cholesterol bilayers as self-assembled amphiphile block polymers that contain headgroups
Xiaoyuan Wang, Fredric S. Cohen, Shixin Xu +1
Cholesterol is known to modulate the structure and function of biological membranes. In this study, we use self-consistent field theory (SCFT) to investigate phospholipid/cholester…
A Minimal Control Family of Dynamical Systems for Universal Approximation
Yifei Duan, Yongqiang Cai
The universal approximation property (UAP) holds a fundamental position in deep learning, as it provides a theoretical foundation for the expressive power of neural networks. It is…
Retrieval Backward Attention without Additional Training: Enhance Embeddings of Large Language Models via Repetition
Yifei Duan, Raphael Shang, Deng Liang +1
Language models can be viewed as functions that embed text into Euclidean space, where the quality of the embedding vectors directly determines model performance, training such neu…