149 citations · 280 across the 6 of their papers we have counts for
4 papers · 1 filter
Fine-Tuning Pre-trained Transformers into Decaying Fast Weights
Huanru Henry Mao
Autoregressive Transformers are strong language models but incur O(T) complexity during per-token generation due to the self-attention mechanism. Recent work proposes kernel-based…
A Survey on Self-supervised Pre-training for Sequential Transfer Learning in Neural Networks
Huanru Henry Mao
Deep neural networks are typically trained under a supervised learning framework where a model learns a single task using labeled data. Instead of relying solely on labeled data, p…
ReZero is All You Need: Fast Convergence at Large Depth
Thomas Bachlechner, Bodhisattwa Prasad Majumder, Huanru Henry Mao +2
Deep networks often suffer from vanishing or exploding gradients due to inefficient signal propagation, leading to long training times or convergence difficulties. Various architec…
Improving Neural Story Generation by Targeted Common Sense Grounding
Huanru Henry Mao, Bodhisattwa Prasad Majumder, Julian McAuley +1
Stories generated with neural language models have shown promise in grammatical and stylistic consistency. However, the generated stories are still lacking in common sense reasonin…