3 papers
cs.LG2024
Length independent generalization bounds for deep SSM architectures via Rademacher contraction and stability constraints
Dániel Rácz, Mihály Petreczky, Bálint Daróczy
Many state-of-the-art models trained on long-range sequences, for example S4, S5 or LRU, are made of sequential blocks combining State-Space Models (SSMs) with neural networks. In…
cs.LG2023
Optimization dependent generalization bound for ReLU networks based on sensitivity in the tangent bundle
Dániel Rácz, Mihály Petreczky, András Csertán +1
Recent advances in deep learning have given us some very promising results on the generalization ability of deep neural networks, however literature still lacks a comprehensive the…
cs.LG2021
Gradient representations in ReLU networks as similarity functions
Dániel Rácz, Bálint Daróczy
Feed-forward networks can be interpreted as mappings with linear decision surfaces at the level of the last layer. We investigate how the tangent space of the network can be exploi…