6 papers
Customizing the Inductive Biases of Softmax Attention using Structured Matrices
Yilun Kuang, Noah Amsel, Sanae Lotfi +3
The core component of attention is the scoring function, which transforms the inputs into low-dimensional queries and keys and takes the dot product of each pair. While the low-dim…
Diverse Dictionary Learning
Yujia Zheng, Zijian Li, Shunxing Fan +2
Given only observational data , where both the latent variables and the generating process are unknown, recovering is ill-posed without additional assumptions…
Time-Aware Prior Fitted Networks for Zero-Shot Forecasting with Exogenous Variables
Andres Potapczynski, Ravi Kiran Selvam, Tatiana Konstantinova +9
In many time series forecasting settings, the target time series is accompanied by exogenous covariates, such as promotions and prices in retail demand; temperature in energy load;…
Zero-shot Forecasting by Simulation Alone
Boris N. Oreshkin, Mayank Jauhari, Ravi Kiran Selvam +10
Zero-shot time-series forecasting holds great promise, but is still in its infancy, hindered by limited and biased data corpora, leakage-prone evaluation, and privacy and licensing…
A More Realistic Evaluation of Cross-Frequency Transfer Learning and Foundation Forecasting Models
Kin G. Olivares, Malcolm Wolff, Tatiana Konstantinova +8
Cross-frequency transfer learning (CFTL) has emerged as a popular framework for curating large-scale time series datasets to pre-train foundation forecasting models (FFMs). Althoug…
Training Flexible Models of Genetic Variant Effects from Functional Annotations using Accelerated Linear Algebra
Alan N. Amin, Andres Potapczynski, Andrew Gordon Wilson
To understand how genetic variants in human genomes manifest in phenotypes -- traits like height or diseases like asthma -- geneticists have sequenced and measured hundreds of thou…