7 papers
Floating-Point Networks with Automatic Differentiation Can Represent Almost All Floating-Point Functions and Their Gradients
Sejun Park, Yeachan Park, Geonho Hwang
Theoretical studies show that for any differentiable function on a compact domain, there exists a neural network that approximates both the function values and gradients. However,…
Uniform Stability and Generalization Error of GD and SGD on Fixed-Point Parameters
Jonghyun Shin, Sejun Park
We analyze generalization error, uniform stability, and uniform argument stability of gradient descent (GD) and stochastic gradient descent (SGD) over discrete parameter spaces, wh…
Expressive Power of Floating-Point Neural Networks with Arbitrary Reduction Orders and Inexact Activation Implementations
Yeachan Park, Geonho Hwang, Wonyeol Lee +1
Most existing expressivity theories for neural networks assume exact real arithmetic, whereas practical neural networks are executed under finite-precision floating-point arithmeti…
On the Expressive Power of Floating-Point Transformers
Sejun Park, Yeachan Park, Geonho Hwang
The study on the expressive power of transformers shows that transformers are permutation equivariant, and they can approximate all permutation-equivariant continuous functions on…
On Expressive Power of Quantized Neural Networks under Fixed-Point Arithmetic
Yeachan Park, Sejun Park, Geonho Hwang
Existing works on the expressive power of neural networks typically assume real parameters and exact operations. In this work, we study the expressive power of quantized networks u…
Floating-Point Neural Networks Are Provably Robust Universal Approximators
Geonho Hwang, Wonyeol Lee, Yeachan Park +2
The classical universal approximation (UA) theorem for neural networks establishes mild conditions under which a feedforward neural network can approximate a continuous function $f…