5 papers
MICA: Multivariate Infini Compressive Attention for Time Series Forecasting
Willa Potosnak, Nina Å»ukowska, MichaÅ WiliÅski +4
Multivariate forecasting with Transformers faces a core scalability challenge: modeling cross-channel dependencies via attention compounds attention's quadratic sequence complexity…
Seeing Through Circuits: Faithful Mechanistic Interpretability for Vision Transformers
Nina Żukowska, Wolfgang Stammer, Bernt Schiele +1
Transparency of neural networks' internal reasoning is at the heart of interpretability research, adding to trust, safety, and understanding of these models. The field of mechanist…
Investigating Compositional Reasoning in Time Series Foundation Models
Willa Potosnak, Cristian Challu, Mononito Goswami +4
Large pre-trained time series foundation models (TSFMs) have demonstrated promising zero-shot performance across a wide range of domains. However, a question remains: Do TSFMs succ…
Exploring Representations and Interventions in Time Series Foundation Models
MichaÅ WiliÅski, Mononito Goswami, Willa Potosnak +2
Time series foundation models (TSFMs) promise to be powerful tools for a wide range of applications. However, their internal representations and learned concepts are still not well…
Implicit Reasoning in Deep Time Series Forecasting
Willa Potosnak, Cristian Challu, Mononito Goswami +3
Recently, time series foundation models have shown promising zero-shot forecasting performance on time series from a wide range of domains. However, it remains unclear whether thei…