50 citations · 213 across the 25 of their papers we have counts for
25 papers
S2 Reducer: High-Performance Sparse Communication to Accelerate Distributed Deep Learning
Keshi Ge, Yongquan Fu, Zhiquan Lai +2
Distributed stochastic gradient descent (SGD) approach has been widely used in large-scale deep learning, and the gradient collective method is vital to ensure the training scalabi…
Training Deep Neural Networks with Adaptive Momentum Inspired by the Quadratic Optimization
Tao Sun, Huaming Ling, Zuoqiang Shi +2
Heavy ball momentum is crucial in accelerating (stochastic) gradient-based optimization algorithms for machine learning. Existing heavy ball momentum is usually weighted by a unifo…
Deep Tiny Network for Recognition-Oriented Face Image Quality Assessment
Baoyun Peng, Min Liu, Zhaoning Zhang +2
Face recognition has made significant progress in recent years due to deep convolutional neural networks (CNN). In many face recognition (FR) scenarios, face images are acquired fr…
Graph Pooling via Coarsened Graph Infomax
Yunsheng Pang, Yunxiang Zhao, Dongsheng Li
Graph pooling that summaries the information in a large graph into a compact form is essential in hierarchical graph representation learning. Existing graph pooling methods either…
Decentralized Federated Averaging
Tao Sun, Dongsheng Li, Bao Wang
Federated averaging (FedAvg) is a communication efficient algorithm for the distributed training with an enormous number of clients. In FedAvg, clients keep their data locally for…
Inertial Proximal Deep Learning Alternating Minimization for Efficient Neutral Network Training
Linbo Qiao, Tao Sun, Hengyue Pan +1
In recent years, the Deep Learning Alternating Minimization (DLAM), which is actually the alternating minimization applied to the penalty form of the deep neutral networks training…
Meta-Learning for Neural Relation Classification with Distant Supervision
Zhenzhen Li, Jian-Yun Nie, Benyou Wang +4
Distant supervision provides a means to create a large number of weakly labeled data at low cost for relation classification. However, the resulting labeled instances are very nois…
Adaptive Temporal Difference Learning with Linear Function Approximation
Tao Sun, Han Shen, Tianyi Chen +1
This paper revisits the temporal difference (TD) learning algorithm for the policy evaluation tasks in reinforcement learning. Typically, the performance of TD(0) and TD() is ve…
Towards Precise End-to-end Weakly Supervised Object Detection Network
Ke Yang, Dongsheng Li, Yong Dou
It is challenging for weakly supervised object detection network to precisely predict the positions of the objects, since there are no instance-level category annotations. Most exi…
XPipe: Efficient Pipeline Model Parallelism for Multi-GPU DNN Training
Lei Guan, Wotao Yin, Dongsheng Li +1
We propose XPipe, an efficient asynchronous pipeline model parallelism approach for multi-GPU DNN training. XPipe is designed to use multiple GPUs to concurrently and continuously…
General Proximal Incremental Aggregated Gradient Algorithms: Better and Novel Results under General Scheme
Tao Sun, Yuejiao Sun, Dongsheng Li +1
The incremental aggregated gradient algorithm is popular in network optimization and machine learning research. However, the current convergence results require the objective funct…
A Multi-Type Multi-Span Network for Reading Comprehension that Requires Discrete Reasoning
Minghao Hu, Yuxing Peng, Zhen Huang +1
Rapid progress has been made in the field of reading comprehension and question answering, where several systems have achieved human parity in some simplified settings. However, th…
Heavy-ball Algorithms Always Escape Saddle Points
Tao Sun, Dongsheng Li, Zhe Quan +3
Nonconvex optimization algorithms with random initialization have attracted increasing attention recently. It has been showed that many first-order methods always avoid saddle poin…
Retrieve, Read, Rerank: Towards End-to-End Multi-Document Reading Comprehension
Minghao Hu, Yuxing Peng, Zhen Huang +1
This paper considers the reading comprehension task in which multiple documents are given as input. Prior work has shown that a pipeline of retriever, reader, and reranker can impr…
Open-Domain Targeted Sentiment Analysis via Span-Based Extraction and Classification
Minghao Hu, Yuxing Peng, Zhen Huang +2
Open-domain targeted sentiment analysis aims to detect opinion targets along with their sentiment polarities from a sentence. Prior work typically formulates this task as a sequenc…
Locality-Sensitive Sketching for Resilient Network Flow Monitoring
Yongquan Fu, Dongsheng Li, Siqi Shen +2
Network monitoring is vital in modern clouds and data center networks for traffic engineering, network diagnosis, network intrusion detection, which need diverse traffic statistics…
Correlation Congruence for Knowledge Distillation
Baoyun Peng, Xiao Jin, Jiaheng Liu +5
Most teacher-student frameworks based on knowledge distillation (KD) depend on a strong congruent constraint on instance level. However, they usually ignore the correlation between…
Exploring Frame Segmentation Networks for Temporal Action Localization
Ke Yang, Xiaolong Shen, Peng Qiao +3
Temporal action localization is an important task of computer vision. Though many methods have been proposed, it still remains an open question how to predict the temporal location…
Iteratively reweighted penalty alternating minimization methods with continuation for image deblurring
Tao Sun, Dongsheng Li, Hao Jiang +1
In this paper, we consider a class of nonconvex problems with linear constraints appearing frequently in the area of image processing. We solve this problem by the penalty method a…
Non-ergodic Convergence Analysis of Heavy-Ball Algorithms
Tao Sun, Penghang Yin, Dongsheng Li +3
In this paper, we revisit the convergence of the Heavy-ball method, and present improved convergence complexity results in the convex setting. We provide the first non-ergodic O(1/…
Attention-Guided Answer Distillation for Machine Reading Comprehension
Minghao Hu, Yuxing Peng, Furu Wei +4
Despite that current reading comprehension systems have achieved significant advancements, their promising performances are often obtained at the cost of making an ensemble of nume…
An Efficient ADMM-Based Algorithm to Nonconvex Penalized Support Vector Machines
Lei Guan, Linbo Qiao, Dongsheng Li +3
Support vector machines (SVMs) with sparsity-inducing nonconvex penalties have received considerable attentions for the characteristics of automatic classification and variable sel…
Loss Rank Mining: A General Hard Example Mining Method for Real-time Detectors
Hao Yu, Zhaoning Zhang, Zheng Qin +4
Modern object detectors usually suffer from low accuracy issues, as foregrounds always drown in tons of backgrounds and become hard examples during training. Compared with those pr…
Diagonalwise Refactorization: An Efficient Training Method for Depthwise Convolutions
Zheng Qin, Zhaoning Zhang, Dongsheng Li +2
Depthwise convolutions provide significant performance benefits owing to the reduction in both parameters and mult-adds. However, training depthwise convolution layers with GPUs is…