8 citations · 11 across the 4 of their papers we have counts for
4 papers · 1 filter
Pushing Toward the Simplex Vertices: A Simple Remedy for Code Collapse in Smoothed Vector Quantization
Takashi Morita
Vector quantization, which discretizes a continuous vector space into a finite set of representative vectors (a codebook), has been widely adopted in modern machine learning. Despi…
Emergence of the Primacy Effect in Structured State-Space Models
Takashi Morita
Structured state-space models (SSMs) have been developed to offer more persistent memory retention than traditional recurrent neural networks, while maintaining real-time inference…
Oscillations enhance time-series prediction in reservoir computing with feedback
Yuji Kawai, Takashi Morita, Jihoon Park +1
Reservoir computing, a machine learning framework used for modeling the brain, can predict temporal data with little observations and minimal computational resources. However, it i…
Positional Encoding Helps Recurrent Neural Networks Handle a Large Vocabulary
Takashi Morita
This study reports an unintuitive finding that positional encoding enhances learning of recurrent neural networks (RNNs). Positional encoding is a high-dimensional representation o…