279 citations · 952 across the 35 of their papers we have counts for
8 papers · 1 filter
What's Hidden in a Randomly Weighted Neural Network?
Vivek Ramanujan, Mitchell Wortsman, Aniruddha Kembhavi +2
Training a neural network is synonymous with learning the values of the weights. By contrast, we demonstrate that randomly weighted neural networks contain subnetworks which achiev…
DeFINE: DEep Factorized INput Token Embeddings for Neural Sequence Modeling
Sachin Mehta, Rik Koncel-Kedziorski, Mohammad Rastegari +1
For sequence models with large vocabularies, a majority of network parameters lie in the input and output layers. In this work, we describe a new method, DeFINE, for learning deep…
Assisted Excitation of Activations: A Learning Technique to Improve Object Detectors
Mohammad Mahdi Derakhshani, Saeed Masoudnia, Amir Hossein Shaker +4
We present a simple and effective learning technique that significantly improves mAP of YOLO object detectors without compromising their speed. During network training, we carefull…
DiCENet: Dimension-wise Convolutions for Efficient Networks
Sachin Mehta, Hannaneh Hajishirzi, Mohammad Rastegari
We introduce a novel and generic convolutional unit, DiCE unit, that is built using dimension-wise convolutions and dimension-wise fusion. The dimension-wise convolutions apply lig…
Butterfly Transform: An Efficient FFT Based Neural Architecture Design
Keivan Alizadeh Vahid, Anish Prabhu, Ali Farhadi +1
In this paper, we show that extending the butterfly operations from the FFT algorithm to a general Butterfly Transform (BFT) can be beneficial in building an efficient block struct…
Discovering Neural Wirings
Mitchell Wortsman, Ali Farhadi, Mohammad Rastegari
The success of neural networks has driven a shift in focus from feature engineering to architecture engineering. However, successful networks today are constructed using a small an…