150 citations · 445 across the 14 of their papers we have counts for
4 papers · 1 filter
Rephrasing the Web: A Recipe for Compute and Data-Efficient Language Modeling
Pratyush Maini, Skyler Seto, He Bai +3
Large language models are trained on massive scrapes of the web, which are often unstructured, noisy, and poorly phrased. Current scaling laws show that learning from such data req…
Construction of Paired Knowledge Graph-Text Datasets Informed by Cyclic Evaluation
Ali Mousavi, Xin Zhan, He Bai +9
Datasets that pair Knowledge Graphs (KG) and text together (KG-T) can be used to train forward and reverse neural models that generate text from KG and vice versa. However models t…
RNN Approaches to Text Normalization: A Challenge
Richard Sproat, Navdeep Jaitly
This paper presents a challenge to the community: given a large corpus of written text aligned to its normalized spoken form, train an RNN to learn the correct normalization functi…
Very Deep Convolutional Networks for End-to-End Speech Recognition
Yu Zhang, William Chan, Navdeep Jaitly
Sequence-to-sequence models have shown success in end-to-end speech recognition. However these models have only used shallow acoustic encoder networks. In our work, we successively…