54 citations · 105 across the 17 of their papers we have counts for
7 papers · 1 filter
Wavelets Are All You Need for Autoregressive Image Generation
Wael Mattar, Idan Levy, Nir Sharon +1
In this paper, we take a new approach to autoregressive image generation that is based on two main ingredients. The first is wavelet image coding, which allows to tokenize the visu…
Reverse Engineering Self-Supervised Learning
Ido Ben-Shaul, Ravid Shwartz-Ziv, Tomer Galanti +2
Self-supervised learning (SSL) is a powerful tool in machine learning, but understanding the learned representations and their underlying mechanisms remains a challenge. This paper…
Numerical Methods For PDEs Over Manifolds Using Spectral Physics Informed Neural Networks
Yuval Zelig, Shai Dekel
We introduce an approach for solving PDEs over manifolds using physics informed neural networks whose architecture aligns with spectral methods. The networks are trained to take in…
Exploring the Approximation Capabilities of Multiplicative Neural Networks for Smooth Functions
Ido Ben-Shaul, Tomer Galanti, Shai Dekel
Multiplication layers are a key component in various influential neural network modules, including self-attention and hypernetwork layers. In this paper, we investigate the approxi…
Nearest Class-Center Simplification through Intermediate Layers
Ido Ben-Shaul, Shai Dekel
Recent advances in theoretical Deep Learning have introduced geometric properties that occur during training, past the Interpolation Threshold -- where the training error reaches z…
Sparsity-Probe: Analysis tool for Deep Learning Models
Ido Ben-Shaul, Shai Dekel
We propose a probe for the analysis of deep learning architectures that is based on machine learning and approximation theoretical principles. Given a deep learning architecture an…
Wavelet Decomposition of Gradient Boosting
Shai Dekel, Oren Elisha, Ohad Morgan
In this paper we introduce a significant improvement to the popular tree-based Stochastic Gradient Boosting algorithm using a wavelet decomposition of the trees. This approach is b…