7 papers
Interdomain Attention: Beyond Token-Level Key-Value Memory
Naoki Kiyohara, Harrison Bo Hua Zhu, Riccardo El Hassanin +4
Softmax attention-based transformers and recurrent memory-based models (e.g., state space models (SSMs)) form the foundation of modern sequence models. Hybrid architectures aim to…
Compact Memory for Continual Logistic Regression
Yohan Jung, Hyungi Lee, Wenlong Chen +4
Despite recent progress, continual learning still does not match the performance of batch training. To avoid catastrophic forgetting, we need to build compact memory of essential p…
HiBBO: HiPPO-based Space Consistency for High-dimensional Bayesian Optimisation
Junyu Xuan, Wenlong Chen, Yingzhen Li
Bayesian Optimisation (BO) is a powerful tool for optimising expensive blackbox functions but its effectiveness diminishes in highdimensional spaces due to sparse data and poor sur…
Variational Uncertainty Decomposition for In-Context Learning
I. Shavindra Jayasekera, Jacob Si, Filippo Valdettaro +3
As large language models (LLMs) gain popularity in conducting prediction tasks in-context, understanding the sources of uncertainty in in-context learning becomes essential to ensu…
Bayesian Computation in Deep Learning
Wenlong Chen, Bolian Li, Ruqi Zhang +1
Bayesian methods have shown success in deep learning applications. For example, in predictive tasks, Bayesian neural networks leverage Bayesian reasoning of model uncertainty to im…
Recurrent Memory for Online Interdomain Gaussian Processes
Wenlong Chen, Naoki Kiyohara, Harrison Bo Hua Zhu +3
We propose a novel online Gaussian process (GP) model that is capable of capturing long-term memory in sequential data in an online learning setting. Our model, Online HiPPO Sparse…
Your Image is Secretly the Last Frame of a Pseudo Video
Wenlong Chen, Wenlin Chen, Lapo Rastrelli +1
Diffusion models, which can be viewed as a special case of hierarchical variational autoencoders (HVAEs), have shown profound success in generating photo-realistic images. In contr…