184 citations · 267 across the 12 of their papers we have counts for
11 papers · 1 filter
Conciseness: An Overlooked Language Task
Felix Stahlberg, Aashish Kumar, Chris Alberti +1
We report on novel investigations into training models that make sentences concise. We define the task and show that it is different from related tasks such as summarization and si…
Simple and Effective Gradient-Based Tuning of Sequence-to-Sequence Models
Jared Lichtarge, Chris Alberti, Shankar Kumar
Recent trends towards training ever-larger language models have substantially improved machine learning performance across linguistic tasks. However, the huge cost of training larg…
Jam or Cream First? Modeling Ambiguity in Neural Machine Translation with SCONES
Felix Stahlberg, Shankar Kumar
The softmax layer in neural machine translation is designed to model the distribution over mutually exclusive tokens. Machine translation, however, is intrinsically uncertain: the…
Uncertainty Determines the Adequacy of the Mode and the Tractability of Decoding in Sequence-to-Sequence Models
Felix Stahlberg, Ilia Kulikov, Shankar Kumar
In many natural language processing (NLP) tasks the same input (e.g. source sentence) can have multiple possible outputs (e.g. translations). To analyze how this ambiguity (also kn…
Capitalization Normalization for Language Modeling with an Accurate and Efficient Hierarchical RNN Model
Hao Zhang, You-Chi Cheng, Shankar Kumar +3
Capitalization normalization (truecasing) is the task of restoring the correct case (uppercase or lowercase) of noisy text. We propose a fast, accurate and compact two-level hierar…
Position-Invariant Truecasing with a Word-and-Character Hierarchical Recurrent Neural Network
Hao Zhang, You-Chi Cheng, Shankar Kumar +2
Truecasing is the task of restoring the correct case (uppercase or lowercase) of noisy text generated either by an automatic system for speech recognition or machine translation or…