5 citations · 10 across the 11 of their papers we have counts for
13 papers
Phase Transition from Random Features to Fully Trainable Networks
Xin He, Huan Lao, Xiliang Lu +1
Constructing accurate and computationally tractable approximation spaces is a fundamental task in scientific computing. Partially trainable neural networks bridge fixed random feat…
Optimal Tradeoffs Between Network Size and Parameter Magnitude in Neural Approximation and Minimax Regression
Baicheng Li, Zuowei Shen, Haizhao Yang +1
The statistical accuracy of neural networks depends on both their approximation power and the complexity of the class fitted from data. While increasing network size is a natural w…
Layer-wise Geometric Approximation Rates for Deep Networks
Shijun Zhang, Zuowei Shen, Yuesheng Xu
Depth is widely viewed as a central contributor to the success of deep neural networks, whereas standard neural network approximation theory typically provides guarantees only for…
Neural Network Approximation: A View from Polytope Decomposition
ZeYu Li, ShiJun Zhang, TieYong Zeng +1
Universal approximation theory offers a foundational framework to verify neural network expressiveness, enabling principled utilization in real-world applications. However, most ex…
NeuronSeek: On Stability and Expressivity of Task-driven Neurons
Hanyu Pei, Jing-Xiao Liao, Qibin Zhao +4
Drawing inspiration from our human brain that designs different neurons for different tasks, recent advances in deep learning have explored modifying a network's neurons to develop…
Fourier Multi-Component and Multi-Layer Neural Networks: Unlocking High-Frequency Potential
Shijun Zhang, Hongkai Zhao, Yimin Zhong +1
The architecture of a neural network and the choice of its activation function are both fundamental to its performance. Equally important is ensuring that these two elements are we…
Hyper-Compression: Model Compression via Hyperfunction
Fenglei Fan, Juntong Fan, Dayang Wang +5
The rapid growth of large models' size has far outpaced that of computing resources. To bridge this gap, encouraged by the parsimonious relationship between genotype and phenotype…
Don't Fear Peculiar Activation Functions: EUAF and Beyond
Qianchao Wang, Shijun Zhang, Dong Zeng +4
In this paper, we propose a new super-expressive activation function called the Parametric Elementary Universal Activation Function (PEUAF). We demonstrate the effectiveness of PEU…
Structured and Balanced Multi-Component and Multi-Layer Neural Networks
Shijun Zhang, Hongkai Zhao, Yimin Zhong +1
In this work, we propose a balanced multi-component and multi-layer neural network (MMNN) structure to accurately and efficiently approximate functions with complex features, in te…
Why Shallow Networks Struggle to Approximate and Learn High Frequencies
Shijun Zhang, Hongkai Zhao, Yimin Zhong +1
In this work, we present a comprehensive study combining mathematical and computational analysis to explain why a two-layer neural network struggles to handle high frequencies in b…
On Enhancing Expressive Power via Compositions of Single Fixed-Size ReLU Network
Shijun Zhang, Jianfeng Lu, Hongkai Zhao
This paper explores the expressive power of deep neural networks through the framework of function compositions. We demonstrate that the repeated compositions of a single fixed-siz…
Neural Network Architecture Beyond Width and Depth
Zuowei Shen, Haizhao Yang, Shijun Zhang
This paper proposes a new neural network architecture by introducing an additional dimension called height beyond width and depth. Neural network architectures with height, width,…
Deep Network Approximation in Terms of Intrinsic Parameters
Zuowei Shen, Haizhao Yang, Shijun Zhang
One of the arguments to explain the success of deep learning is the powerful approximation capacity of deep neural networks. Such capacity is generally accompanied by the explosive…