10 papers · 1 filter
FedOUI: OUI-Guided Client Weighting for Federated Aggregation
Alberto Fernández-Hernández, Jose I. Mestre, Cristian Pérez-Corral +3
Federated learning usually aggregates client updates using dataset size or gradient-level criteria, while overlooking internal signals about how each client model is organizing its…
OUI as a Structural Observable: Towards an Activation-Centric View of Neural Network Training
Alberto Fernández-Hernández, Jose I. Mestre, Cristian Pérez-Corral +3
Activation functions are what make deep networks expressive: without them, the model collapses to a linear map. Yet we still evaluate training mostly from the outside, through loss…
OUIDecay: Adaptive Layer-wise Weight Decay for CNNs Using Online Activation Patterns
Alberto Fernández-Hernández, Jose I. Mestre, Cristian Pérez-Corral +3
Weight decay remains one of the most widely used regularization mechanisms for training convolutional neural networks, yet it is still commonly applied as a fixed coefficient share…
FedSQ: Optimized Weight Averaging via Fixed Gating
Cristian Pérez-Corral, Jose I. Mestre, Alberto Fernández-Hernández +3
Federated learning (FL) enables collaborative training across organizations without sharing raw data, but it is hindered by statistical heterogeneity (non-i.i.d.\ client data) and…
When Learning Rates Go Wrong: Early Structural Signals in PPO Actor-Critic
Alberto Fernández-Hernández, Cristian Pérez-Corral, Jose I. Mestre +3
Deep Reinforcement Learning systems are highly sensitive to the learning rate (LR), and selecting stable and performant training runs often requires extensive hyperparameter search…
Regime Change Hypothesis: Foundations for Decoupled Dynamics in Neural Network Training
Cristian Pérez-Corral, Alberto Fernández-Hernández, Jose I. Mestre +3
Despite the empirical success of DNN, their internal training dynamics remain difficult to characterize. In ReLU-based models, the activation pattern induced by a given input deter…
GLAI: GreenLightningAI for Accelerated Training through Knowledge Decoupling
Jose I. Mestre, Alberto Fernández-Hernández, Cristian Pérez-Corral +3
In this work we introduce GreenLightningAI (GLAI), a new architectural block designed as an alternative to conventional MLPs. The central idea is to separate two types of knowledge…
Sinusoidal Initialization, Time for a New Start
Alberto Fernández-Hernández, Jose I. Mestre, Manuel F. Dolz +2
Initialization plays a critical role in Deep Neural Network training, directly influencing convergence, stability, and generalization. Common approaches such as Glorot and He initi…
OUI Need to Talk About Weight Decay: A New Perspective on Overfitting Detection
Alberto Fernández-Hernández, Jose I. Mestre, Manuel F. Dolz +2
We introduce the Overfitting-Underfitting Indicator (OUI), a novel tool for monitoring the training dynamics of Deep Neural Networks (DNNs) and identifying optimal regularization h…
GreenLightningAI: An Efficient AI System with Decoupled Structural and Quantitative Knowledge
Jose Duato, Jose I. Mestre, Manuel F. Dolz +1
The number and complexity of artificial intelligence (AI) applications is growing relentlessly. As a result, even with the many algorithmic and mathematical advances experienced ov…