74 citations · 132 across the 12 of their papers we have counts for
12 papers
Empowering Federated Learning for Massive Models with NVIDIA FLARE
Holger R. Roth, Ziyue Xu, Yuan-Ting Hsieh +12
In the ever-evolving landscape of artificial intelligence (AI) and large language models (LLMs), handling and leveraging data effectively has become a critical challenge. Most stat…
Tied-Lora: Enhancing parameter efficiency of LoRA with weight tying
Adithya Renduchintala, Tugrul Konuk, Oleksii Kuchaiev
We introduce Tied-LoRA, a novel paradigm leveraging weight tying and selective training to enhance the parameter efficiency of Low-rank Adaptation (LoRA). Our exploration encompass…
Multilingual Neural Machine Translation with Deep Encoder and Multiple Shallow Decoders
Xiang Kong, Adithya Renduchintala, James Cross +3
Recent work in multilingual translation advances translation quality surpassing bilingual baselines using deep transformer models with increased capacity. However, the extra latenc…
Investigating Failures of Automatic Translation in the Case of Unambiguous Gender
Adithya Renduchintala, Adina Williams
Transformer based models are the modern work horses for neural machine translation (NMT), reaching state of the art across several benchmarks. Despite their impressive accuracy, we…
XLEnt: Mining a Large Cross-lingual Entity Dataset with Lexical-Semantic-Phonetic Word Alignment
Ahmed El-Kishky, Adithya Renduchintala, James Cross +2
Cross-lingual named-entity lexica are an important resource to multilingual NLP tasks such as machine translation and cross-lingual wikification. While knowledge bases contain a la…
Towards Understanding the Behaviors of Optimal Deep Active Learning Algorithms
Yilun Zhou, Adithya Renduchintala, Xian Li +3
Active learning (AL) algorithms may achieve better performance with fewer data because the model guides the data selection process. While many algorithms have been proposed, there…
Quality Estimation without Human-labeled Data
Yi-Lin Tuan, Ahmed El-Kishky, Adithya Renduchintala +3
Quality estimation aims to measure the quality of translated content without access to a reference translation. This is crucial for machine translation systems in real-world scenar…
A Call for Prudent Choice of Subword Merge Operations in Neural Machine Translation
Shuoyang Ding, Adithya Renduchintala, Kevin Duh
Most neural machine translation systems are built upon subword units extracted by methods such as Byte-Pair Encoding (BPE) or wordpiece. However, the choice of number of merge oper…
Pretraining by Backtranslation for End-to-end ASR in Low-Resource Settings
Matthew Wiesner, Adithya Renduchintala, Shinji Watanabe +3
We explore training attention-based encoder-decoder ASR in low-resource settings. These models perform poorly when trained on small amounts of transcribed speech, in part because t…
Character-Aware Decoder for Translation into Morphologically Rich Languages
Adithya Renduchintala, Pamela Shapiro, Kevin Duh +1
Neural machine translation (NMT) systems operate primarily on words (or sub-words), ignoring lower-level patterns of morphology. We present a character-aware decoder designed to ca…
ESPnet: End-to-End Speech Processing Toolkit
Shinji Watanabe, Takaaki Hori, Shigeki Karita +9
This paper introduces a new open source platform for end-to-end speech processing named ESPnet. ESPnet mainly focuses on end-to-end automatic speech recognition (ASR), and adopts w…
Multi-Modal Data Augmentation for End-to-End ASR
Adithya Renduchintala, Shuoyang Ding, Matthew Wiesner +1
We present a new end-to-end architecture for automatic speech recognition (ASR) that can be trained using \emph{symbolic} input in addition to the traditional acoustic input. This…