activity
20152024
most citedAlexa Teacher Model: Pretraining and Distilling Multi-Billion-Parameter Encoders for Natural Language Understanding Systems

64 citations · 185 across the 9 of their papers we have counts for

collaborators

9 papers

cs.CL2024★ 2 cited

When Benchmarks are Targets: Revealing the Sensitivity of Large Language Model Leaderboards

Norah Alzahrani, Hisham Abdullah Alyahya, Yazeed Alnumay +9

Large Language Model (LLM) leaderboards based on benchmark rankings are regularly used to guide practitioners in model selection. Often, the published leaderboard rankings are take…

cs.CL2023★ 8 cited

Translation-Enhanced Multilingual Text-to-Image Generation

Yaoyiran Li, Ching-Yun Chang, Stephen Rawls +2

Research on text-to-image generation (TTI) still predominantly focuses on the English language due to the lack of annotated image-caption data in other languages; in the long run,…

cs.CV2023★ 7 cited

Scalable and Accurate Self-supervised Multimodal Representation Learning without Aligned Video and Text Data

Vladislav Lialin, Stephen Rawls, David Chan +3

Scaling up weakly-supervised datasets has shown to be highly effective in the image-text domain and has contributed to most of the recent state-of-the-art computer vision and multi…

cs.CL2022★ 38 cited

AlexaTM 20B: Few-Shot Learning Using a Large-Scale Multilingual Seq2Seq Model

Saleh Soltan, Shankar Ananthakrishnan, Jack FitzGerald +13

In this work, we demonstrate that multilingual large-scale sequence-to-sequence (seq2seq) models, pre-trained on a mixture of denoising and Causal Language Modeling (CLM) tasks, ar…

cs.CL2022★ 64 cited

Alexa Teacher Model: Pretraining and Distilling Multi-Billion-Parameter Encoders for Natural Language Understanding Systems

Jack FitzGerald, Shankar Ananthakrishnan, Konstantine Arkoudas +38

We present results from a large-scale experiment on pretraining encoders with non-embedding parameter counts ranging from 700M to 9.3B, their subsequent distillation into smaller m…

cs.CL2020★ 1 cited

Don't Parse, Insert: Multilingual Semantic Parsing with Insertion Based Decoding

Qile Zhu, Haidar Khan, Saleh Soltan +2

Semantic parsing is one of the key components of natural language understanding systems. A successful parse transforms an input utterance to an action that is easily understood by…