373 citations · 386 across the 4 of their papers we have counts for
3 papers · 1 filter
Interdomain Attention: Beyond Token-Level Key-Value Memory
Naoki Kiyohara, Harrison Bo Hua Zhu, Riccardo El Hassanin +4
Softmax attention-based transformers and recurrent memory-based models (e.g., state space models (SSMs)) form the foundation of modern sequence models. Hybrid architectures aim to…
Neural Stochastic Flows: Solver-Free Modelling and Inference for SDE Solutions
Naoki Kiyohara, Edward Johns, Yingzhen Li
Stochastic differential equations (SDEs) are well suited to modelling noisy and irregularly sampled time series found in finance, physics, and machine learning. Traditional approac…
Recurrent Memory for Online Interdomain Gaussian Processes
Wenlong Chen, Naoki Kiyohara, Harrison Bo Hua Zhu +3
We propose a novel online Gaussian process (GP) model that is capable of capturing long-term memory in sequential data in an online learning setting. Our model, Online HiPPO Sparse…