Showing cs.LGShow all
2 papers · 1 filter
cs.LG2022
A Combinatorial Perspective on the Optimization of Shallow ReLU Networks
Michael Matena, Colin Raffel
The NP-hard problem of optimizing a shallow ReLU network can be characterized as a combinatorial search over each training example's activation pattern followed by a constrained co…
cs.LG2021
Do Transformer Modifications Transfer Across Implementations and Applications?
Sharan Narang, Hyung Won Chung, Yi Tay +13
The research community has proposed copious modifications to the Transformer architecture since it was introduced over three years ago, relatively few of which have seen widespread…