60 citations · 191 across the 23 of their papers we have counts for
23 papers
Beyond Discreteness: Sample Complexity Analysis of Straight-Through Estimator for 1-bit Quantization
Halyun Jeong, Jack Xin, Penghang Yin
Training quantized neural networks requires addressing the non-differentiable and discrete nature of the underlying optimization problem. To tackle this challenge, the straight-thr…
COMQ: A Backpropagation-Free Algorithm for Post-Training Quantization
Aozhong Zhang, Zi Yang, Naigang Wang +4
Post-training quantization (PTQ) has emerged as a practical approach to compress large neural networks, making them highly efficient for deployment. However, effectively reducing t…
Feature Affinity Assisted Knowledge Distillation and Quantization of Deep Neural Networks on Label-Free Data
Zhijian Li, Biao Yang, Penghang Yin +2
In this paper, we propose a feature affinity (FA) assisted knowledge distillation (KD) method to improve quantization-aware training of deep neural networks (DNN). The FA loss on i…
Recurrence of Optimum for Training Weight and Activation Quantized Networks
Ziang Long, Penghang Yin, Jack Xin
Deep neural networks (DNNs) are quantized for efficient inference on resource-constrained platforms. However, training deep learning models with low-precision weights and activatio…
Learning Quantized Neural Nets by Coarse Gradient Method for Non-linear Classification
Ziang Long, Penghang Yin, Jack Xin
Quantized or low-bit neural networks are attractive due to their inference efficiency. However, training deep neural networks with quantized activations involves minimizing a disco…
Global Convergence and Geometric Characterization of Slow to Fast Weight Evolution in Neural Network Training for Classifying Linearly Non-Separable Data
Ziang Long, Penghang Yin, Jack Xin
In this paper, we study the dynamics of gradient descent in learning neural networks for classification problems. Unlike in existing works, we consider the linearly non-separable c…
Understanding Straight-Through Estimator in Training Activation Quantized Neural Nets
Penghang Yin, Jiancheng Lyu, Shuai Zhang +3
Training activation quantized neural networks involves minimizing a piecewise constant function whose gradient vanishes almost everywhere, which is undesirable for the standard bac…
Non-ergodic Convergence Analysis of Heavy-Ball Algorithms
Tao Sun, Penghang Yin, Dongsheng Li +3
In this paper, we revisit the convergence of the Heavy-ball method, and present improved convergence complexity results in the convex setting. We provide the first non-ergodic O(1/…
Adversarial Defense via Data Dependent Activation Function and Total Variation Minimization
Bao Wang, Alex T. Lin, Wei Zhu +3
We improve the robustness of Deep Neural Net (DNN) to adversarial attacks by using an interpolating function as the output activation. This data-dependent activation remarkably imp…
Blended Coarse Gradient Descent for Full Quantization of Deep Neural Networks
Penghang Yin, Shuai Zhang, Jiancheng Lyu +3
Quantized deep neural networks (QDNNs) are attractive due to their much lower memory storage and faster inference speed than their regular full precision counterparts. To maintain…
Laplacian Smoothing Gradient Descent
Stanley Osher, Bao Wang, Penghang Yin +4
We propose a class of very simple modifications of gradient descent and stochastic gradient descent. We show that when applied to a large variety of machine learning problems, rang…
Generalized Proximal Smoothing for Phase Retrieval
Minh Pham, Penghang Yin, Arjun Rana +2
In this paper, we report the development of the generalized proximal smoothing (GPS) algorithm for phase retrieval of noisy data. GPS is a optimization-based algorithm, in which we…
Non-ergodic Complexity of Convex Proximal Inertial Gradient Descents
Tao Sun, Linbo Qiao, Dongsheng Li
The proximal inertial gradient descent is efficient for the composite minimization and applicable for broad of machine learning problems. In this paper, we revisit the computationa…
BinaryRelax: A Relaxation Approach For Training Deep Neural Networks With Quantized Weights
Penghang Yin, Shuai Zhang, Jiancheng Lyu +3
We propose BinaryRelax, a simple two-phase algorithm, for training deep neural networks with quantized weights. The set constraint that characterizes the quantization of weights is…
Deep Learning for Real-Time Crime Forecasting and its Ternarization
Bao Wang, Penghang Yin, Andrea L. Bertozzi +3
Real-time crime forecasting is important. However, accurate prediction of when and where the next crime will happen is difficult. No known physical model provides a reasonable appr…
Stochastic Backward Euler: An Implicit Gradient Descent Algorithm for -means Clustering
Penghang Yin, Minh Pham, Adam Oberman +1
In this paper, we propose an implicit gradient descent algorithm for the classic -means problem. The implicit gradient step or backward Euler is solved via stochastic fixed-poin…
Computations of optimal transport distance with Fisher information regularization
Wuchen Li, Penghang Yin, Stanley Osher
We propose a fast algorithm to approximate the optimal transport distance. The main idea is to add a Fisher information regularization into the dynamical setting of the problem, or…
-minimization method for link flow correction
Penghang Yin, Zhe Sun, Wenlong Jin +1
A computational method, based on -minimization, is proposed for the problem of link flow correction, when the available traffic flow data on many links in a road network ar…
Quantization and Training of Low Bit-Width Convolutional Neural Networks for Object Detection
Penghang Yin, Shuai Zhang, Yingyong Qi +1
We present LBW-Net, an efficient optimization based method for quantization and training of the low bit-width convolutional neural networks (CNNs). Specifically, we quantize the we…
Iterative minimization for non-convex compressed sensing
Penghang Yin, Jack Xin
An algorithmic framework, based on the difference of convex functions algorithm (DCA), is proposed for minimizing a class of concave sparse metrics for compressed sensing problems.…
Transformed Schatten-1 Iterative Thresholding Algorithms for Low Rank Matrix Completion
Shuai Zhang, Penghang Yin, Jack Xin
We study a non-convex low-rank promoting penalty function, the transformed Schatten-1 (TS1), and its applications in matrix completion. The TS1 penalty, as a matrix quasi-norm defi…
PhaseLiftOff: an Accurate and Stable Phase Retrieval Method Based on Difference of Trace and Frobenius Norms
Penghang Yin, Jack Xin
Phase retrieval aims to recover a signal from its amplitude measurements , , where 's are over-complete basis vectors, with…
A Geometric Blind Source Separation Method Based on Facet Component Analysis
P. Yin, Y. Sun, J. Xin
Given a set of mixtures, blind source separation attempts to retrieve the source signals without or with very little information of the the mixing process. We present a geometric a…