163 citations · 304 across the 9 of their papers we have counts for
3 papers · 1 filter
Small Character Models Match Large Word Models for Autocomplete Under Memory Constraints
Ganesh Jawahar, Subhabrata Mukherjee, Debadeepta Dey +5
Autocomplete is a task where the user inputs a piece of text, termed prompt, which is conditioned by the model to generate semantically coherent continuation. Existing works for th…
One Network Doesn't Rule Them All: Moving Beyond Handcrafted Architectures in Self-Supervised Learning
Sharath Girish, Debadeepta Dey, Neel Joshi +5
The current literature on self-supervised learning (SSL) focuses on developing learning objectives to train neural networks more effectively on unlabeled data. The typical developm…
LiteTransformerSearch: Training-free Neural Architecture Search for Efficient Language Models
Mojan Javaheripi, Gustavo H. de Rosa, Subhabrata Mukherjee +6
The Transformer architecture is ubiquitously used as the building block of large-scale autoregressive language models. However, finding architectures with the optimal trade-off bet…