4 papers
Extracurricular Learning: Knowledge Transfer Beyond Empirical Distribution
Hadi Pouransari, Mojan Javaheripi, Vinay Sharma +1
Knowledge distillation has been used to transfer knowledge learned by a sophisticated model (teacher) to a simpler model (student). This technique is widely used to compress model…
Least squares binary quantization of neural networks
Hadi Pouransari, Zhucheng Tu, Oncel Tuzel
Quantizing weights and activations of deep neural networks results in significant improvement in inference efficiency at the cost of lower accuracy. A source of the accuracy gap be…
Democratizing Production-Scale Distributed Deep Learning
Minghuang Ma, Hadi Pouransari, Daniel Chao +5
The interest and demand for training deep neural networks have been experiencing rapid growth, spanning a wide range of applications in both academia and industry. However, trainin…
A distributed-memory hierarchical solver for general sparse linear systems
Chao Chen, Hadi Pouransari, Sivasankaran Rajamanickam +2
We present a parallel hierarchical solver for general sparse linear systems on distributed-memory machines. For large-scale problems, this fully algebraic algorithm is faster and m…