118 citations · 809 across the 58 of their papers we have counts for
16 papers · 1 filter
HAWQV3: Dyadic Neural Network Quantization
Zhewei Yao, Zhen Dong, Zhangcheng Zheng +8
Current low-precision quantization algorithms often have the hidden cost of conversion back and forth from floating point to quantized integer values. This hidden cost limits the l…
A Statistical Framework for Low-bitwidth Training of Deep Neural Networks
Jianfei Chen, Yu Gai, Zhewei Yao +2
Fully quantized training (FQT), which uses low-bitwidth hardware by quantizing the activations, weights, and gradients of a neural network model, is a promising approach to acceler…
MAF: Multimodal Alignment Framework for Weakly-Supervised Phrase Grounding
Qinxin Wang, Hao Tan, Sheng Shen +2
Phrase localization is a task that studies the mapping from textual phrases to regions of an image. Given difficulties in annotating phrase-to-object datasets at scale, we develop…
Training Recommender Systems at Scale: Communication-Efficient Model and Data Parallelism
Vipul Gupta, Dhruv Choudhary, Ping Tak Peter Tang +6
In this paper, we consider hybrid parallelism -- a paradigm that employs both Data Parallelism (DP) and Model Parallelism (MP) -- to scale distributed training of large recommendat…
Continuous-in-Depth Neural Networks
Alejandro F. Queiruga, N. Benjamin Erichson, Dane Taylor +1
Recent work has attempted to interpret residual networks (ResNets) as one step of a forward Euler discretization of an ordinary differential equation, focusing mainly on syntactic…
Improving Semi-supervised Federated Learning by Reducing the Gradient Diversity of Models
Zhengming Zhang, Yaoqing Yang, Zhewei Yao +3
Federated learning (FL) is a promising way to use the computing power of mobile devices while maintaining the privacy of users. Current work in FL, however, makes the unrealistic a…
Noise-Response Analysis of Deep Neural Networks Quantifies Robustness and Fingerprints Structural Malware
N. Benjamin Erichson, Dane Taylor, Qixuan Wu +1
The ubiquity of deep neural networks (DNNs), cloud-based training, and transfer learning is giving rise to a new cybersecurity frontier in which unsecure DNNs have `structural malw…
Adversarially-Trained Deep Nets Transfer Better: Illustration on Image Classification
Francisco Utrera, Evan Kravitz, N. Benjamin Erichson +2
Transfer learning has emerged as a powerful methodology for adapting pre-trained deep neural networks on image recognition tasks to new domains. This process consists of taking a n…
Boundary thickness and robustness in learning models
Yaoqing Yang, Rajiv Khanna, Yaodong Yu +5
Robustness of machine learning models to various adversarial and non-adversarial corruptions continues to be of interest. In this paper, we introduce the notion of the boundary thi…
Lipschitz Recurrent Neural Networks
N. Benjamin Erichson, Omri Azencot, Alejandro Queiruga +2
Viewing recurrent neural networks (RNNs) as continuous-time dynamical systems, we propose a recurrent unit that describes the hidden state's evolution with two parts: a well-unders…
Precise expressions for random projections: Low-rank approximation and randomized Newton
Michał Dereziński, Feynman Liang, Zhenyu Liao +1
It is often desirable to reduce the dimensionality of a large dataset by projecting it onto a low-dimensional subspace. Matrix sketching has emerged as a powerful technique for per…
ADAHESSIAN: An Adaptive Second Order Optimizer for Machine Learning
Zhewei Yao, Amir Gholami, Sheng Shen +3
We introduce ADAHESSIAN, a second order stochastic optimization algorithm which dynamically incorporates the curvature of the loss function via ADAptive estimates of the HESSIAN. S…
Determinantal Point Processes in Randomized Numerical Linear Algebra
Michał Dereziński, Michael W. Mahoney
Randomized Numerical Linear Algebra (RandNLA) uses randomness to develop improved algorithms for matrix problems that arise in scientific computing, data science, machine learning,…
Flow-based Algorithms for Improving Clusters: A Unifying Framework, Software, and Performance
K. Fountoulakis, M. Liu, D. F. Gleich +1
Clustering points in a vector space or nodes in a graph is a ubiquitous primitive in statistical data analysis, and it is commonly used for exploratory data analysis. In practice,…
Error Estimation for Sketched SVD via the Bootstrap
Miles E. Lopes, N. Benjamin Erichson, Michael W. Mahoney
In order to compute fast approximations to the singular value decompositions (SVD) of very large matrices, randomized sketching algorithms have become a leading approach. However,…
ZeroQ: A Novel Zero Shot Quantization Framework
Yaohui Cai, Zhewei Yao, Zhen Dong +3
Quantization is a promising approach for reducing the inference time and memory footprint of neural networks. However, most existing quantization methods require access to the orig…