15 citations · 17 across the 4 of their papers we have counts for
4 papers
Towards A Unified View of Sparse Feed-Forward Network in Pretraining Large Language Model
Zeyu Leo Liu, Tim Dettmers, Xi Victoria Lin +2
Large and sparse feed-forward layers (S-FFN) such as Mixture-of-Experts (MoE) have proven effective in scaling up Transformers model size for \textit{pretraining} large language mo…
Training Large-Vocabulary Neural Language Models by Private Federated Learning for Resource-Constrained Devices
Mingbin Xu, Congzheng Song, Ye Tian +10
Federated Learning (FL) is a technique to train models using data distributed across devices. Differential Privacy (DP) provides a formal privacy guarantee for sensitive data. Our…
Voice trigger detection from LVCSR hypothesis lattices using bidirectional lattice recurrent neural networks
Woojay Jeon, Leo Liu, Henry Mason
We propose a method to reduce false voice triggers of a speech-enabled personal assistant by post-processing the hypothesis lattice of a server-side large-vocabulary continuous spe…
SNDCNN: Self-normalizing deep CNNs with scaled exponential linear units for speech recognition
Zhen Huang, Tim Ng, Leo Liu +3
Very deep CNNs achieve state-of-the-art results in both computer vision and speech recognition, but are difficult to train. The most popular way to train very deep CNNs is to use s…