1.1k citations · 1.6k across the 34 of their papers we have counts for
16 papers · 1 filter
Evo: Evolving the Evolver through Weight-Harness Co-Evolution
Chejian Xu, Shujian Zhang, Wenxuan Zhou +5
Self-evolving agents are increasingly used to improve language model systems without human intervention through either weight updates or harness improvements. Existing approaches t…
Steering LLMs for Culturally Localized Generation
Simran Khanuja, Hongbin Liu, Shujian Zhang +4
LLMs are deployed globally, yet produce responses biased towards cultures with abundant training data. Existing cultural localization approaches such as prompting or post-training…
MUSIC: MUlti-Step Instruction Contrast for Multi-Turn Reward Models
Wenzhe Li, Shujian Zhang, Wenxuan Zhou +5
Evaluating the quality of multi-turn conversations is crucial for developing capable Large Language Models (LLMs), yet remains a significant challenge, often requiring costly human…
Fantastic Reasoning Behaviors and Where to Find Them: Unsupervised Discovery of the Reasoning Process
Zhenyu Zhang, Shujian Zhang, John Lambert +6
Despite the growing reasoning capabilities of recent large language models (LLMs), their internal mechanisms during the reasoning process remain underexplored. Prior approaches oft…
Eliciting Behaviors in Multi-Turn Conversations
Jing Huang, Shujian Zhang, Lun Wang +3
Identifying specific and often complex behaviors from large language models (LLMs) in conversational settings is crucial for their evaluation. Recent work proposes novel techniques…
The Gift of Feedback: Improving ASR Model Quality by Learning from User Corrections through Federated Learning
Lillian Zhou, Yuxin Ding, Mingqing Chen +5
Automatic speech recognition (ASR) models are typically trained on large datasets of transcribed speech. As language evolves and new terms come into use, these models can become ou…
Large vocabulary speech recognition for languages of Africa: multilingual modeling and self-supervised learning
Sandy Ritchie, You-Chi Cheng, Mingqing Chen +4
Almost none of the 2,000+ languages spoken in Africa have widely available automatic speech recognition systems, and the required data is also only available for a few languages. W…
Scaling Language Model Size in Cross-Device Federated Learning
Jae Hun Ro, Theresa Breiner, Lara McConnaughey +4
Most studies in cross-device federated learning focus on small models, due to the server-client communication and on-device computation bottlenecks. In this work, we leverage vario…
Detecting Unintended Memorization in Language-Model-Fused ASR
W. Ronny Huang, Steve Chien, Om Thakkar +1
End-to-end (E2E) models are often being accompanied by language models (LMs) via shallow fusion for boosting their overall quality as well as recognition of rare words. At the same…
Capitalization Normalization for Language Modeling with an Accurate and Efficient Hierarchical RNN Model
Hao Zhang, You-Chi Cheng, Shankar Kumar +3
Capitalization normalization (truecasing) is the task of restoring the correct case (uppercase or lowercase) of noisy text. We propose a fast, accurate and compact two-level hierar…
Position-Invariant Truecasing with a Word-and-Character Hierarchical Recurrent Neural Network
Hao Zhang, You-Chi Cheng, Shankar Kumar +2
Truecasing is the task of restoring the correct case (uppercase or lowercase) of noisy text generated either by an automatic system for speech recognition or machine translation or…
A Method to Reveal Speaker Identity in Distributed ASR Training, and How to Counter It
Trung Dang, Om Thakkar, Swaroop Ramaswamy +3
End-to-end Automatic Speech Recognition (ASR) models are commonly trained over spoken utterances using optimization methods like Stochastic Gradient Descent (SGD). In distributed s…
Federated Learning of N-gram Language Models
Mingqing Chen, Ananda Theertha Suresh, Rajiv Mathews +4
We propose algorithms to train production-quality n-gram language models using federated learning. Federated learning is a distributed computation platform that can be used to trai…
Federated Learning for Emoji Prediction in a Mobile Keyboard
Swaroop Ramaswamy, Rajiv Mathews, Kanishka Rao +1
We show that a word-level recurrent neural network can predict emoji from text typed on a mobile keyboard. We demonstrate the usefulness of transfer learning for predicting emoji b…
Federated Learning Of Out-Of-Vocabulary Words
Mingqing Chen, Rajiv Mathews, Tom Ouyang +1
We demonstrate that a character-level recurrent neural network is able to learn out-of-vocabulary (OOV) words under federated learning settings, for the purpose of expanding the vo…
Federated Learning for Mobile Keyboard Prediction
Andrew Hard, Kanishka Rao, Rajiv Mathews +6
We train a recurrent neural network language model using a distributed, on-device learning framework called federated learning for the purpose of next-word prediction in a virtual…