149 citations · 280 across the 6 of their papers we have counts for
8 papers
Fine-Tuning Pre-trained Transformers into Decaying Fast Weights
Huanru Henry Mao
Autoregressive Transformers are strong language models but incur O(T) complexity during per-token generation due to the self-attention mechanism. Recent work proposes kernel-based…
A Survey on Self-supervised Pre-training for Sequential Transfer Learning in Neural Networks
Huanru Henry Mao
Deep neural networks are typically trained under a supervised learning framework where a model learns a single task using labeled data. Instead of relying solely on labeled data, p…
Speech Recognition and Multi-Speaker Diarization of Long Conversations
Huanru Henry Mao, Shuyang Li, Julian McAuley +1
Speech recognition (ASR) and speaker diarization (SD) models have traditionally been trained separately to produce rich conversation transcripts with speaker labels. Recent advance…
ReZero is All You Need: Fast Convergence at Large Depth
Thomas Bachlechner, Bodhisattwa Prasad Majumder, Huanru Henry Mao +2
Deep networks often suffer from vanishing or exploding gradients due to inefficient signal propagation, leading to long training times or convergence difficulties. Various architec…
Improving Neural Story Generation by Targeted Common Sense Grounding
Huanru Henry Mao, Bodhisattwa Prasad Majumder, Julian McAuley +1
Stories generated with neural language models have shown promise in grammatical and stylistic consistency. However, the generated stories are still lacking in common sense reasonin…
LakhNES: Improving multi-instrumental music generation with cross-domain pre-training
Chris Donahue, Huanru Henry Mao, Yiting Ethan Li +2
We are interested in the task of generating multi-instrumental music scores. The Transformer architecture has recently shown great promise for the task of piano score generation; h…
The NES Music Database: A multi-instrumental dataset with expressive performance attributes
Chris Donahue, Huanru Henry Mao, Julian McAuley
Existing research on music generation focuses on composition, but often ignores the expressive performance characteristics required for plausible renditions of resultant pieces. In…
DeepJ: Style-Specific Music Generation
Huanru Henry Mao, Taylor Shin, Garrison W. Cottrell
Recent advances in deep neural networks have enabled algorithms to compose music that is comparable to music composed by humans. However, few algorithms allow the user to generate…