11 citations · 44 across the 18 of their papers we have counts for
8 papers · 1 filter
Mitigating Unintended Memorization in Language Models via Alternating Teaching
Zhe Liu, Xuedong Zhang, Fuchun Peng
Recent research has shown that language models have a tendency to memorize rare or unique sequences in the training corpora which can thus leak sensitive attributes of user data. W…
Neural-FST Class Language Model for End-to-End Speech Recognition
Antoine Bruguier, Duc Le, Rohit Prabhavalkar +7
We propose Neural-FST Class Language Model (NFCLM) for end-to-end speech recognition, a novel method that combines neural network language models (NNLMs) and finite state transduce…
Federated Marginal Personalization for ASR Rescoring
Zhe Liu, Fuchun Peng
We introduce federated marginal personalization (FMP), a novel method for continuously updating personalized neural network language models (NNLMs) on private devices using federat…
Improving N-gram Language Models with Pre-trained Deep Transformer
Yiren Wang, Hongzhao Huang, Zhe Liu +4
Although n-gram language models (LMs) have been outperformed by the state-of-the-art neural LMs, they are still widely used in speech recognition due to its high efficiency in infe…
An Empirical Study of Efficient ASR Rescoring with Transformers
Hongzhao Huang, Fuchun Peng
Neural language models (LMs) have been proved to significantly outperform classical n-gram LMs for language modeling due to their superior abilities to model long-range dependencie…
Efficient Dynamic WFST Decoding for Personalized Language Models
Jun Liu, Jiedan Zhu, Vishal Kathuria +1
We propose a two-layer cache mechanism to speed up dynamic WFST decoding with personalized language models. The first layer is a public cache that stores most of the static part of…
Training ASR models by Generation of Contextual Information
Kritika Singh, Dmytro Okhonko, Jun Liu +8
Supervised ASR models have reached unprecedented levels of accuracy, thanks in part to ever-increasing amounts of labelled training data. However, in many applications and locales,…
Analyzing the Forgetting Problem in the Pretrain-Finetuning of Dialogue Response Models
Tianxing He, Jun Liu, Kyunghyun Cho +4
In this work, we study how the finetuning stage in the pretrain-finetune framework changes the behavior of a pretrained neural language generator. We focus on the transformer encod…