37 citations · 143 across the 30 of their papers we have counts for
31 papers
The Power of External Memory in Increasing Predictive Model Capacity
Cenk Baykal, Dylan J Cutler, Nishanth Dikkala +3
One way of introducing sparsity into deep networks is by attaching an external table of parameters that is sparsely looked up at different layers of the network. By storing the bul…
Alternating Updates for Efficient Transformers
Cenk Baykal, Dylan Cutler, Nishanth Dikkala +3
It has been well established that increasing scale in deep transformer networks leads to improved quality and performance. However, this increase in scale often comes with prohibit…
A Theoretical View on Sparsely Activated Networks
Cenk Baykal, Nishanth Dikkala, Rina Panigrahy +2
Deep and wide neural networks successfully fit very complex functions today, but dense models are starting to be prohibitively expensive for inference. To mitigate this, one promis…
A Unified Cascaded Encoder ASR Model for Dynamic Model Sizes
Shaojin Ding, Weiran Wang, Ding Zhao +11
In this paper, we propose a dynamic cascaded encoder Automatic Speech Recognition (ASR) model, which unifies models for different deployment scenarios. Moreover, the model can sign…
Provable Hierarchical Lifelong Learning with a Sketch-based Modular Architecture
Zihao Deng, Zee Fryer, Brendan Juba +2
We propose a modular architecture for the lifelong learning of hierarchically structured tasks. Specifically, we prove that our architecture is theoretically able to learn tasks th…
One Network Fits All? Modular versus Monolithic Task Formulations in Neural Networks
Atish Agarwala, Abhimanyu Das, Brendan Juba +4
Can deep learning solve multiple tasks simultaneously, even when they are unrelated and very different? We investigate how the representations of the underlying tasks affect the ab…