3 citations · 7 across the 22 of their papers we have counts for
14 papers · 1 filter
Algorithmic Foundations of Deep Learning: Complexity-Theoretic Rates and a Characterization of Universal Approximation
Anastasis Kratsios, Simone Brugiapaglia, Bum Jun Kim +2
Feedforward neural network (NN) expressivity is typically studied by emulating optimal basis-expansion schemes. While powerful, this perspective is incomplete: it primarily capture…
k-Maximum Inner Product Attention for Graph Transformers and the Expressive Power of GraphGPS
Jonas De Schouwer, Haitz Sáez de Ocáriz Borde, Xiaowen Dong
Graph transformers have shown promise in overcoming limitations of traditional graph neural networks, such as oversquashing and difficulties in modeling long-range dependencies. Ho…
Assessing the Geographic Generalization and Physical Consistency of Generative Models for Climate Downscaling
Carlo Saccardi, Maximilian Pierzyna, Haitz Sáez de Ocáriz Borde +6
Kilometer-scale weather data is crucial for real-world applications but remains computationally intensive to produce using traditional weather simulations. An emerging solution is…
Bridging Graph and State-Space Modeling for Intensive Care Unit Length of Stay Prediction
Shuqi Zi, Haitz Sáez de Ocáriz Borde, Emma Rocheteau +1
Predicting a patient's length of stay (LOS) in the intensive care unit (ICU) is a critical task for hospital resource management, yet remains challenging due to the heterogeneous a…
Mathematical Foundations of Geometric Deep Learning
Haitz Sáez de Ocáriz Borde, Michael Bronstein
We review the key mathematical concepts necessary for studying Geometric Deep Learning.
LoRA Fine-Tuning Without GPUs: A CPU-Efficient Meta-Generation Framework for LLMs
Reza Arabpour, Haitz Sáez de Ocáriz Borde, Anastasis Kratsios
Low-Rank Adapters (LoRAs) have transformed the fine-tuning of Large Language Models (LLMs) by enabling parameter-efficient updates. However, their widespread adoption remains limit…
Beyond Parallelism: Synergistic Computational Graph Effects in Multi-Head Attention
Haitz Sáez de Ocáriz Borde
Multi-head attention powers Transformer networks, the primary deep learning architecture behind the success of large language models (LLMs). Yet, the theoretical advantages of mult…
Towards Quantifying Long-Range Interactions in Graph Machine Learning: a Large Graph Dataset and a Measurement
Huidong Liang, Haitz Sáez de Ocáriz Borde, Baskaran Sripathmanathan +2
Long-range dependencies are critical for effective graph representation learning, yet most existing datasets focus on small graphs tailored to inductive tasks, offering limited ins…
Scalable Message Passing Neural Networks: No Need for Attention in Large Graph Representation Learning
Haitz Sáez de Ocáriz Borde, Artem Lukoianov, Anastasis Kratsios +2
We propose Scalable Message Passing Neural Networks (SMPNNs) and demonstrate that, by integrating standard convolutional message passing into a Pre-Layer Normalization Transformer-…
Neural Spacetimes for DAG Representation Learning
Haitz Sáez de Ocáriz Borde, Anastasis Kratsios, Marc T. Law +2
We propose a class of trainable deep learning-based geometries called Neural Spacetimes (NSTs), which can universally represent nodes in weighted directed acyclic graphs (DAGs) as…
Asymmetry in Low-Rank Adapters of Foundation Models
Jiacheng Zhu, Kristjan Greenewald, Kimia Nadjahi +6
Parameter-efficient fine-tuning optimizes large, pre-trained foundation models by updating a subset of parameters; in this class, Low-Rank Adaptation (LoRA) is particularly effecti…
AMES: A Differentiable Embedding Space Selection Framework for Latent Graph Inference
Yuan Lu, Haitz Sáez de Ocáriz Borde, Pietro Liò
In real-world scenarios, although data entities may possess inherent relationships, the specific graph illustrating their connections might not be directly accessible. Latent graph…
Neural Snowflakes: Universal Latent Graph Inference via Trainable Latent Geometries
Haitz Sáez de Ocáriz Borde, Anastasis Kratsios
The inductive bias of a graph neural network (GNN) is largely encoded in its specified graph. Latent graph inference relies on latent geometric representations to dynamically rewir…
Capacity Bounds for Hyperbolic Neural Network Representations of Latent Tree Structures
Anastasis Kratsios, Ruiyang Hong, Haitz Sáez de Ocáriz Borde
We study the representation capacity of deep hyperbolic neural networks (HNNs) with a ReLU activation function. We establish the first proof that HNNs can -isometrical…