4 papers
Neural Attention Search Linear: Towards Adaptive Token-Level Hybrid Attention Models
Difan Deng, Andreas Bentzen Winje, Lukas Fehring +1
The quadratic computational complexity of softmax transformers has become a bottleneck in long-context scenarios. In contrast, linear attention model families provide a promising d…
Neural Attention Search
Difan Deng, Marius Lindauer
We present Neural Attention Search (NAtS), a framework that automatically evaluates the importance of each token within a sequence and determines if the corresponding token can be…
Optimizing Time Series Forecasting Architectures: A Hierarchical Neural Architecture Search Approach
Difan Deng, Marius Lindauer
The rapid development of time series forecasting research has brought many deep learning-based modules in this field. However, despite the increasing amount of new forecasting arch…
auto-sktime: Automated Time Series Forecasting
Marc-André Zöller, Marius Lindauer, Marco F. Huber
In today's data-driven landscape, time series forecasting is pivotal in decision-making across various sectors. Yet, the proliferation of more diverse time series data, coupled wit…