5 citations · 10 across the 11 of their papers we have counts for
10 papers · 1 filter
Layer-wise Geometric Approximation Rates for Deep Networks
Shijun Zhang, Zuowei Shen, Yuesheng Xu
Depth is widely viewed as a central contributor to the success of deep neural networks, whereas standard neural network approximation theory typically provides guarantees only for…
Neural Network Approximation: A View from Polytope Decomposition
ZeYu Li, ShiJun Zhang, TieYong Zeng +1
Universal approximation theory offers a foundational framework to verify neural network expressiveness, enabling principled utilization in real-world applications. However, most ex…
NeuronSeek: On Stability and Expressivity of Task-driven Neurons
Hanyu Pei, Jing-Xiao Liao, Qibin Zhao +4
Drawing inspiration from our human brain that designs different neurons for different tasks, recent advances in deep learning have explored modifying a network's neurons to develop…
Fourier Multi-Component and Multi-Layer Neural Networks: Unlocking High-Frequency Potential
Shijun Zhang, Hongkai Zhao, Yimin Zhong +1
The architecture of a neural network and the choice of its activation function are both fundamental to its performance. Equally important is ensuring that these two elements are we…
Hyper-Compression: Model Compression via Hyperfunction
Fenglei Fan, Juntong Fan, Dayang Wang +5
The rapid growth of large models' size has far outpaced that of computing resources. To bridge this gap, encouraged by the parsimonious relationship between genotype and phenotype…
Structured and Balanced Multi-Component and Multi-Layer Neural Networks
Shijun Zhang, Hongkai Zhao, Yimin Zhong +1
In this work, we propose a balanced multi-component and multi-layer neural network (MMNN) structure to accurately and efficiently approximate functions with complex features, in te…
Why Shallow Networks Struggle to Approximate and Learn High Frequencies
Shijun Zhang, Hongkai Zhao, Yimin Zhong +1
In this work, we present a comprehensive study combining mathematical and computational analysis to explain why a two-layer neural network struggles to handle high frequencies in b…
On Enhancing Expressive Power via Compositions of Single Fixed-Size ReLU Network
Shijun Zhang, Jianfeng Lu, Hongkai Zhao
This paper explores the expressive power of deep neural networks through the framework of function compositions. We demonstrate that the repeated compositions of a single fixed-siz…
Neural Network Architecture Beyond Width and Depth
Zuowei Shen, Haizhao Yang, Shijun Zhang
This paper proposes a new neural network architecture by introducing an additional dimension called height beyond width and depth. Neural network architectures with height, width,…
Deep Network Approximation in Terms of Intrinsic Parameters
Zuowei Shen, Haizhao Yang, Shijun Zhang
One of the arguments to explain the success of deep learning is the powerful approximation capacity of deep neural networks. Such capacity is generally accompanied by the explosive…