Balancing Expressivity and Robustness: Constrained Rational Activations for Reinforcement Learning

Rafał Surdej, Michał Bortkiewicz, Alex Lewandowski +2

Trainable activation functions, whose parameters are optimized alongside network weights, offer increased expressivity compared to fixed activation functions. Specifically, trainab…