3 citations · 6 across the 5 of their papers we have counts for
5 papers
Fine-Tuning BERT with Character-Level Noise for Zero-Shot Transfer to Dialects and Closely-Related Languages
Aarohi Srivastava, David Chiang
In this work, we induce character-level noise in various forms when fine-tuning BERT to enable zero-shot cross-lingual transfer to unseen dialects and languages. We fine-tune BERT…
Tighter Bounds on the Expressivity of Transformer Encoders
David Chiang, Peter Cholak, Anand Pillay
Characterizing neural networks in terms of better-understood formal systems has the potential to yield new insights into the power and limitations of these networks. Doing so for t…
A Continuum of Generation Tasks for Investigating Length Bias and Degenerate Repetition
Darcey Riley, David Chiang
Language models suffer from various degenerate behaviors. These differ between tasks: machine translation (MT) exhibits length bias, while tasks like story generation exhibit exces…
The Surprising Computational Power of Nondeterministic Stack RNNs
Brian DuSell, David Chiang
Traditional recurrent neural networks (RNNs) have a fixed, finite number of memory cells. In theory (assuming bounded range and precision), this limits their formal language recogn…
Overcoming a Theoretical Limitation of Self-Attention
David Chiang, Peter Cholak
Although transformers are remarkably effective for many tasks, there are some surprisingly easy-looking regular languages that they struggle with. Hahn shows that for languages whe…