61 citations · 67 across the 3 of their papers we have counts for
7 papers
Measuring and Improving Model-Moderator Collaboration using Uncertainty Estimation
Ian D. Kivlichan, Zi Lin, Jeremiah Liu +1
Content moderation is often performed by a collaboration between humans and machine learning models. However, it is not well understood how to design the collaborative process so a…
Pruning Redundant Mappings in Transformer Models via Spectral-Normalized Identity Prior
Zi Lin, Jeremiah Zhe Liu, Zi Yang +2
Traditional (unstructured) pruning methods for a Transformer model focus on regularizing the individual weights by penalizing them toward zero. In this work, we explore spectral-no…
Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awareness
Jeremiah Zhe Liu, Zi Lin, Shreyas Padhy +3
Bayesian neural networks (BNN) and deep ensembles are principled approaches to estimate the predictive uncertainty of a deep learning model. However their practicality in real-time…
Fast Structured Decoding for Sequence Models
Zhiqing Sun, Zhuohan Li, Haoqing Wang +3
Autoregressive sequence models achieve state-of-the-art performance in domains like machine translation. However, due to the autoregressive factorization nature, these models suffe…
Hint-Based Training for Non-Autoregressive Machine Translation
Zhuohan Li, Zi Lin, Di He +4
Due to the unparallelizable nature of the autoregressive factorization, AutoRegressive Translation (ART) models have to generate tokens sequentially during decoding and thus suffer…
Implanting Rational Knowledge into Distributed Representation at Morpheme Level
Zi Lin, Yang Liu
Previously, researchers paid no attention to the creation of unambiguous morpheme embeddings independent from the corpus, while such information plays an important role in expressi…