7 papers · 1 filter
Interdomain Attention: Beyond Token-Level Key-Value Memory
Naoki Kiyohara, Harrison Bo Hua Zhu, Riccardo El Hassanin +4
Transformers and deep state space models (SSMs) sit at opposite ends of a basic design choice: attention routes each query through a growing key-value (KV) cache by content-based m…
Probabilistic Learning and Generation in Deep Sequence Models
Wenlong Chen
Despite exceptional predictive performance of Deep sequence models (DSMs), the main concern of their deployment centers around the lack of uncertainty awareness. In contrast, proba…
Compact Memory for Continual Logistic Regression
Yohan Jung, Hyungi Lee, Wenlong Chen +4
Despite recent progress, continual learning still does not match the performance of batch training. To avoid catastrophic forgetting, we need to build compact memory of essential p…
Bayesian Computation in Deep Learning
Wenlong Chen, Bolian Li, Ruqi Zhang +1
Bayesian methods have shown success in deep learning applications. For example, in predictive tasks, Bayesian neural networks leverage Bayesian reasoning of model uncertainty to im…
HiBBO: HiPPO-based Space Consistency for High-dimensional Bayesian Optimisation
Junyu Xuan, Wenlong Chen, Yingzhen Li
Bayesian Optimisation (BO) is a powerful tool for optimising expensive blackbox functions but its effectiveness diminishes in highdimensional spaces due to sparse data and poor sur…
Recurrent Memory for Online Interdomain Gaussian Processes
Wenlong Chen, Naoki Kiyohara, Harrison Bo Hua Zhu +3
We propose a novel online Gaussian process (GP) model that is capable of capturing long-term memory in sequential data in an online learning setting. Our model, Online HiPPO Sparse…