1 citations · 1 across the 14 of their papers we have counts for
12 papers · 1 filter
Early Memory Selection for Balanced Adam
Alberto Fernández-Hernández, Cristian Pérez-Corral, Jose I. Mestre +2
We propose a method for choosing the shared memory parameter in Adam from a short pilot training. The selected remains fixed during the subsequent full training. A…
Detecting Atypical Clients in Federated Learning via Representation-Level Divergence
Cristian Pérez-Corral, Jose I. Mestre, Alberto Fernández-Hernández +2
Federated learning enables collaborative training across distributed clients with heterogeneous data, but such heterogeneity often leads to unstable updates and degraded global per…
StableGrad: Backward Scale Control without Batch Normalization
Jose I. Mestre, Alberto Fernández-Hernández, Cristian Pérez-Corral +2
Training very deep neural networks requires controlling the propagation of magnitudes across depth. Without such control, activations and gradients may vanish, explode, or enter un…
FedOUI: OUI-Guided Client Weighting for Federated Aggregation
Alberto Fernández-Hernández, Jose I. Mestre, Cristian Pérez-Corral +3
Federated learning usually aggregates client updates using dataset size or gradient-level criteria, while overlooking internal signals about how each client model is organizing its…
OUI as a Structural Observable: Towards an Activation-Centric View of Neural Network Training
Alberto Fernández-Hernández, Jose I. Mestre, Cristian Pérez-Corral +3
Activation functions are what make deep networks expressive: without them, the model collapses to a linear map. Yet we still evaluate training mostly from the outside, through loss…
Refresh-Scaling the Memory of Balanced Adam
Alberto Fernández-Hernández, Cristian Pérez-Corral, Jose I. Mestre +2
Recent evidence suggests that Adam performs robustly when its momentum parameters are tied, , reducing the optimizer to a single remaining parameter. However, how this par…
OUIDecay: Adaptive Layer-wise Weight Decay for CNNs Using Online Activation Patterns
Alberto Fernández-Hernández, Jose I. Mestre, Cristian Pérez-Corral +3
Weight decay remains one of the most widely used regularization mechanisms for training convolutional neural networks, yet it is still commonly applied as a fixed coefficient share…
FedSQ: Optimized Weight Averaging via Fixed Gating
Cristian Pérez-Corral, Jose I. Mestre, Alberto Fernández-Hernández +3
Federated learning (FL) enables collaborative training across organizations without sharing raw data, but it is hindered by statistical heterogeneity (non-i.i.d.\ client data) and…
-GELU: Learning Gating Hardness for Controlled ReLU-ization in Deep Networks
Cristian Pérez-Corral, Alberto Fernández-Hernández, Jose I. Mestre +2
Gaussian Error Linear Unit (GELU) is a widely used smooth alternative to Rectifier Linear Unit (ReLU), yet many deployment, compression, and analysis toolchains are most naturally…
When Learning Rates Go Wrong: Early Structural Signals in PPO Actor-Critic
Alberto Fernández-Hernández, Cristian Pérez-Corral, Jose I. Mestre +3
Deep Reinforcement Learning systems are highly sensitive to the learning rate (LR), and selecting stable and performant training runs often requires extensive hyperparameter search…
Regime Change Hypothesis: Foundations for Decoupled Dynamics in Neural Network Training
Cristian Pérez-Corral, Alberto Fernández-Hernández, Jose I. Mestre +3
Despite the empirical success of DNN, their internal training dynamics remain difficult to characterize. In ReLU-based models, the activation pattern induced by a given input deter…
Why Is Dynamically Special in Adam
Alberto Fernández-Hernández, Cristian Pérez-Corral, Jose I. Mestre +2
Adam has been at the core of large-scale training for almost a decade, yet the role of its two momentum parameters remains poorly understood. Recent work shows that tying $β_{1}=β_…