492 citations · 735 across the 11 of their papers we have counts for
Showing 2020Show all
2 papers · 1 filter
cs.CL2020★ 4 cited
Pruning Redundant Mappings in Transformer Models via Spectral-Normalized Identity Prior
Zi Lin, Jeremiah Zhe Liu, Zi Yang +2
Traditional (unstructured) pruning methods for a Transformer model focus on regularizing the individual weights by penalizing them toward zero. In this work, we explore spectral-no…
cs.LG2020★ 103 cited
Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awareness
Jeremiah Zhe Liu, Zi Lin, Shreyas Padhy +3
Bayesian neural networks (BNN) and deep ensembles are principled approaches to estimate the predictive uncertainty of a deep learning model. However their practicality in real-time…