45 citations · 93 across the 25 of their papers we have counts for
3 papers · 1 filter
Attention in a family of Boltzmann machines emerging from modern Hopfield networks
Toshihiro Ota, Ryo Karakida
Hopfield networks and Boltzmann machines (BMs) are fundamental energy-based neural network models. Recent studies on modern Hopfield networks have broaden the class of energy funct…
Understanding Gradient Regularization in Deep Learning: Efficient Finite-Difference Computation and Implicit Bias
Ryo Karakida, Tomoumi Takase, Tomohiro Hayase +1
Gradient regularization (GR) is a method that penalizes the gradient norm of the training loss during training. While some studies have reported that GR can improve generalization…
Deep Learning in Random Neural Fields: Numerical Experiments via Neural Tangent Kernel
Kaito Watanabe, Kotaro Sakamoto, Ryo Karakida +2
A biological neural network in the cortex forms a neural field. Neurons in the field have their own receptive fields, and connection weights between two neurons are random but high…