3 citations · 6 across the 9 of their papers we have counts for
4 papers · 1 filter
Multi-armed bandits for resource efficient, online optimization of language model pre-training: the use case of dynamic masking
Iñigo Urteaga, Moulay-Zaïdane Draïdia, Tomer Lancewicki +1
We design and evaluate a Bayesian optimization framework for resource efficient pre-training of Transformer-based language models (TLMs). TLM pre-training requires high computation…
Towards Reinforcement Learning for Pivot-based Neural Machine Translation with Non-autoregressive Transformer
Evgeniia Tokarchuk, Jan Rosendahl, Weiyue Wang +4
Pivot-based neural machine translation (NMT) is commonly used in low-resource setups, especially for translation between non-English language pairs. It benefits from using high res…
Integrated Training for Sequence-to-Sequence Models Using Non-Autoregressive Transformer
Evgeniia Tokarchuk, Jan Rosendahl, Weiyue Wang +4
Complex natural language applications such as speech translation or pivot translation traditionally rely on cascaded models. However, cascaded models are known to be prone to error…
Deploying a BERT-based Query-Title Relevance Classifier in a Production System: a View from the Trenches
Leonard Dahlmann, Tomer Lancewicki
The Bidirectional Encoder Representations from Transformers (BERT) model has been radically improving the performance of many Natural Language Processing (NLP) tasks such as Text C…