2.5k citations · 2.5k across the 4 of their papers we have counts for
Showing cs.NEShow all
3 papers · 1 filter
cs.NE2026
Mixture of Experts with Soft Nearest Neighbor Loss: Resolving Expert Collapse via Representation Disentanglement
Abien Fred Agarap, Arnulfo P. Azcarraga
The Mixture-of-Experts (MoE) model uses a set of expert networks that specialize on subsets of a dataset under the supervision of a gating network. A common issue in MoE architectu…
cs.NE2024★ 2 cited
k-Winners-Take-All Ensemble Neural Network
Abien Fred Agarap, Arnulfo P. Azcarraga
Ensembling is one approach that improves the performance of a neural network by combining a number of independent neural networks, usually by either averaging or summing up their i…
cs.NE2018★ 2.5k cited
Deep Learning using Rectified Linear Units (ReLU)
Abien Fred Agarap
The Rectified Linear Unit (ReLU) is a foundational activation function in artficial neural networks. Recent literature frequently misattributes its origin to the 2018 (initial) ver…