198 citations · 775 across the 43 of their papers we have counts for
43 papers
Mirror Natural Evolution Strategies
Haishan Ye
The zeroth-order optimization has been widely used in machine learning applications. However, the theoretical study of the zeroth-order optimization focus on the algorithms which a…
IDEA: Interpretable Dynamic Ensemble Architecture for Time Series Prediction
Mengyue Zha, Kani Chen, Tong Zhang
We enhance the accuracy and generalization of univariate time series point prediction by an explainable ensemble on the fly. We propose an Interpretable Dynamic Ensemble Architectu…
Efficient Neural Network Training via Forward and Backward Propagation Sparsification
Xiao Zhou, Weizhong Zhang, Zonghao Chen +2
Sparse training is a natural idea to accelerate the training speed of deep neural networks and save the memory usage, especially since large modern neural networks are significantl…
When is the Convergence Time of Langevin Algorithms Dimension Independent? A Composite Optimization Viewpoint
Yoav Freund, Yi-An Ma, Tong Zhang
There has been a surge of works bridging MCMC sampling and optimization, with a specific focus on translating non-asymptotic convergence guarantees for optimization problems into t…
Feel-Good Thompson Sampling for Contextual Bandits and Reinforcement Learning
Tong Zhang
Thompson Sampling has been widely used for contextual bandit problems due to the flexibility of its modeling power. However, a general theory for this class of methods in the frequ…
Eigencurve: Optimal Learning Rate Schedule for SGD on Quadratic Objectives with Skewed Hessian Spectrums
Rui Pan, Haishan Ye, Tong Zhang
Learning rate schedulers have been widely adopted in training deep neural networks. Despite their practical importance, there is a discrepancy between its practice and its theoreti…
Feature Correlation Aggregation: on the Path to Better Graph Neural Networks
Jieming Zhou, Tong Zhang, Pengfei Fang +2
Prior to the introduction of Graph Neural Networks (GNNs), modeling and analyzing irregular data, particularly graphs, was thought to be the Achilles' heel of deep learning. The co…
Near Optimal Stochastic Algorithms for Finite-Sum Unbalanced Convex-Concave Minimax Optimization
Luo Luo, Guangzeng Xie, Tong Zhang +1
This paper considers stochastic first-order algorithms for convex-concave minimax problems of the form , where can be presented by the…
KECRS: Towards Knowledge-Enriched Conversational Recommendation System
Tong Zhang, Yong Liu, Peixiang Zhong +3
The chit-chat-based conversational recommendation systems (CRS) provide item recommendations to users through natural language interactions. To better understand user's intentions,…
Effective Sparsification of Neural Networks with Global Sparsity Constraint
Xiao Zhou, Weizhong Zhang, Hang Xu +1
Weight pruning is an effective technique to reduce the model size and inference time for deep neural networks in real-world deployments. However, since magnitudes and relative impo…
Reinforced Attention for Few-Shot Learning and Beyond
Jie Hong, Pengfei Fang, Weihao Li +4
Few-shot learning aims to correctly recognize query samples from unseen classes given a limited number of support samples, often by relying on global embeddings of images. In this…
DeEPCA: Decentralized Exact PCA with Linear Convergence Rate
Haishan Ye, Tong Zhang
Due to the rapid growth of smart agents such as weakly connected computational nodes and sensors, developing decentralized algorithms that can perform computations on local agents…
Mathematical Models of Overparameterized Neural Networks
Cong Fang, Hanze Dong, Tong Zhang
Deep learning has received considerable empirical successes in recent years. However, while many ad hoc tricks have been discovered by practitioners, until recently, there has been…
Multi-modal AsynDGAN: Learn From Distributed Medical Image Data without Sharing Private Information
Qi Chang, Zhennan Yan, Lohendran Baskaran +5
As deep learning technologies advance, increasingly more data is necessary to generate general and robust models for various tasks. In the medical domain, however, large-scale and…
PMGT-VR: A decentralized proximal-gradient algorithmic framework with variance reduction
Haishan Ye, Wei Xiong, Tong Zhang
This paper considers the decentralized composite optimization problem. We propose a novel decentralized variance-reduction proximal-gradient algorithmic framework, called PMGT-VR,…
DiffMG: Differentiable Meta Graph Search for Heterogeneous Graph Neural Networks
Yuhui Ding, Quanming Yao, Huan Zhao +1
In this paper, we propose a novel framework to automatically utilize task-dependent semantic information which is encoded in heterogeneous information networks (HINs). Specifically…
Error Compensated Distributed SGD Can Be Accelerated
Xun Qian, Peter Richtárik, Tong Zhang
Gradient compression is a recent and increasingly popular technique for reducing the communication cost in distributed training of large-scale machine learning models. In this work…
Modeling from Features: a Mean-field Framework for Over-parameterized Deep Neural Networks
Cong Fang, Jason D. Lee, Pengkun Yang +1
This paper proposes a new mean-field framework for over-parameterized deep neural networks (DNNs), which can be used to analyze neural network training. In this framework, a DNN is…
Guided Learning of Nonconvex Models through Successive Functional Gradient Optimization
Rie Johnson, Tong Zhang
This paper presents a framework of successive functional gradient optimization for training nonconvex models such as neural networks, where training is driven by mirror descent in…
Synthetic Learning: Learn From Distributed Asynchronized Discriminator GAN Without Sharing Medical Image Data
Qi Chang, Hui Qu, Yikai Zhang +4
In this paper, we propose a data privacy-preserving and communication efficient distributed GAN learning framework named Distributed Asynchronized Discriminator GAN (AsynDGAN). Our…
Multi-consensus Decentralized Accelerated Gradient Descent
Haishan Ye, Luo Luo, Ziang Zhou +1
This paper considers the decentralized convex optimization problem, which has a wide range of applications in large-scale machine learning, sensor networks, and control theory. We…
MiLeNAS: Efficient Neural Architecture Search via Mixed-Level Reformulation
Chaoyang He, Haishan Ye, Li Shen +1
Many recently proposed methods for Neural Architecture Search (NAS) can be formulated as bilevel optimization. For efficient implementation, its solution requires approximations of…
Bidirectional Generative Modeling Using Adversarial Gradient Estimation
Xinwei Shen, Tong Zhang, Kani Chen
This paper considers the general -divergence formulation of bidirectional generative modeling, which includes VAE and BiGAN as special cases. We present a new optimization metho…
A Generalized Neural Tangent Kernel Analysis for Two-layer Neural Networks
Zixiang Chen, Yuan Cao, Quanquan Gu +1
A recent breakthrough in deep learning theory shows that the training of over-parameterized deep neural networks can be characterized by a kernel function called \textit{neural tan…