10 papers
Interdomain Attention: Beyond Token-Level Key-Value Memory
Naoki Kiyohara, Harrison Bo Hua Zhu, Riccardo El Hassanin +4
Transformers and deep state space models (SSMs) sit at opposite ends of a basic design choice: attention routes each query through a growing key-value (KV) cache by content-based m…
Saliency-Aware Regularized Quantization Calibration for Large Language Models
Yanlong Zhao, Xiaoyuan Cheng, Huihang Liu +6
Post-training quantization (PTQ) is an effective approach for deploying large language models (LLMs) under memory and latency constraints. Most existing PTQ methods determine quant…
Probabilistic Learning and Generation in Deep Sequence Models
Wenlong Chen
Despite exceptional predictive performance of Deep sequence models (DSMs), the main concern of their deployment centers around the lack of uncertainty awareness. In contrast, proba…
Variational Uncertainty Decomposition for In-Context Learning
I. Shavindra Jayasekera, Jacob Si, Filippo Valdettaro +3
As large language models (LLMs) gain popularity in conducting prediction tasks in-context, understanding the sources of uncertainty in in-context learning becomes essential to ensu…
Compact Memory for Continual Logistic Regression
Yohan Jung, Hyungi Lee, Wenlong Chen +4
Despite recent progress, continual learning still does not match the performance of batch training. To avoid catastrophic forgetting, we need to build compact memory of essential p…
Bayesian Computation in Deep Learning
Wenlong Chen, Bolian Li, Ruqi Zhang +1
Bayesian methods have shown success in deep learning applications. For example, in predictive tasks, Bayesian neural networks leverage Bayesian reasoning of model uncertainty to im…