363 citations · 559 across the 15 of their papers we have counts for
4 papers · 1 filter
AlexaTM 20B: Few-Shot Learning Using a Large-Scale Multilingual Seq2Seq Model
Saleh Soltan, Shankar Ananthakrishnan, Jack FitzGerald +13
In this work, we demonstrate that multilingual large-scale sequence-to-sequence (seq2seq) models, pre-trained on a mixture of denoising and Causal Language Modeling (CLM) tasks, ar…
Alexa Teacher Model: Pretraining and Distilling Multi-Billion-Parameter Encoders for Natural Language Understanding Systems
Jack FitzGerald, Shankar Ananthakrishnan, Konstantine Arkoudas +38
We present results from a large-scale experiment on pretraining encoders with non-embedding parameter counts ranging from 700M to 9.3B, their subsequent distillation into smaller m…
Unfreeze with Care: Space-Efficient Fine-Tuning of Semantic Parsing Models
Weiqi Sun, Haidar Khan, Nicolas Guenon des Mesnards +2
Semantic parsing is a key NLP task that maps natural language to structured meaning representations. As in many other NLP tasks, SOTA performance in semantic parsing is now attaine…
RescoreBERT: Discriminative Speech Recognition Rescoring with BERT
Liyan Xu, Yile Gu, Jari Kolehmainen +5
Second-pass rescoring is an important component in automatic speech recognition (ASR) systems that is used to improve the outputs from a first-pass decoder by implementing a lattic…