4 papers · 1 filter
RunningTensor: Generalizing Linear Attention to Higher-Order Recurrent States
Luca Herranz-Celotti, Vincent Guigue
Linear attention and state-space models provide linear-time sequence modeling, but their recurrent memory remains a second-order tensor (a matrix), limiting the order of interactio…
Semantic Space Search Trajectory Networks
Julian Agudelo, Alberto Tonda, Gabriela Ochoa +3
Search Trajectory Networks (STNs) are a graph-based tool for visualizing and characterizing the behavior of optimization algorithms. STNs' reliance on discretization of the search…
Drought forecasting using a hybrid neural architecture for integrating time series and static data
Julian Agudelo, Vincent Guigue, Cristina Manfredotti +1
Reliable forecasting is critical for early warning systems and adaptive drought management. Most previous deep learning approaches focus solely on homogeneous regions and rely on s…
Improving generalization in large language models by learning prefix subspaces
Louis Falissard, Vincent Guigue, Laure Soulier
This article focuses on large language models (LLMs) fine-tuning in the scarce data regime (also known as the "few-shot" learning setting). We propose a method to increase the gene…