399 citations · 3.4k across the 95 of their papers we have counts for
97 papers
Persia: An Open, Hybrid System Scaling Deep Learning-based Recommenders up to 100 Trillion Parameters
Xiangru Lian, Binhang Yuan, Xuefeng Zhu +24
Deep learning based models have dominated the current landscape of production recommender systems. Furthermore, recent years have witnessed an exponential growth of the model scale…
Resilient Consensus-based Multi-agent Reinforcement Learning with Function Approximation
Martin Figura, Yixuan Lin, Ji Liu +1
Adversarial attacks during training can strongly influence the performance of multi-agent reinforcement learning algorithms. It is, thus, highly desirable to augment existing algor…
Joint Channel and Weight Pruning for Model Acceleration on Moblie Devices
Tianli Zhao, Xi Sheryl Zhang, Wentao Zhu +4
For practical deep neural network design on mobile devices, it is essential to consider the constraints incurred by the computational resources and the inference latency in various…
Finite-Time Error Bounds for Distributed Linear Stochastic Approximation
Yixuan Lin, Vijay Gupta, Ji Liu
This paper considers a novel multi-agent linear stochastic approximation algorithm driven by Markovian noise and general consensus-type interaction, in which each agent evolves acc…
HeterPS: Distributed Deep Learning With Reinforcement Learning Based Scheduling in Heterogeneous Environments
Ji Liu, Zhihua Wu, Dianhai Yu +6
Deep neural networks (DNNs) exploit many layers and a large number of parameters to achieve excellent performance. The training process of DNN models generally handles large-scale…
Deep Keyphrase Completion
Yu Zhao, Jia Song, Huali Feng +4
Keyphrase provides accurate information of document content that is highly compact, concise, full of meanings, and widely used for discourse comprehension, organization, and text r…
Multi-Layer SIS Model with an Infrastructure Network
Philip E. Pare, Axel Janson, Sebin Gracy +3
This paper deals with the spread of diseases over both a population network and an infrastructure network. We develop a layered networked spread model for a susceptible-infected-su…
SpeechNAS: Towards Better Trade-off between Latency and Accuracy for Large-Scale Speaker Verification
Wentao Zhu, Tianlong Kong, Shun Lu +6
Recently, x-vector has been a successful and popular approach for speaker verification, which employs a time delay neural network (TDNN) and statistics pooling to extract speaker c…
GDP: Stabilized Neural Network Pruning via Gates with Differentiable Polarization
Yi Guo, Huan Yuan, Jianchao Tan +3
Model compression techniques are recently gaining explosive attention for obtaining efficient AI models for various real-time applications. Channel pruning is one important compres…
POSO: Personalized Cold Start Modules for Large-scale Recommender Systems
Shangfeng Dai, Haobin Lin, Zhichen Zhao +5
Recommendation for new users, also called user cold start, has been a well-recognized challenge for online recommender systems. Most existing methods view the crux as the lack of i…
ChemiRise: a data-driven retrosynthesis engine
Xiangyan Sun, Ke Liu, Yuquan Lin +9
We have developed an end-to-end, retrosynthesis system, named ChemiRise, that can propose complete retrosynthesis routes for organic compounds rapidly and reliably. The system was…
ErrorCompensatedX: error compensation for variance reduced algorithms
Hanlin Tang, Yao Li, Ji Liu +1
Communication cost is one major bottleneck for the scalability for distributed learning. One approach to reduce the communication cost is to compress the gradient during communicat…
Shifted Chunk Transformer for Spatio-Temporal Representational Learning
Xuefan Zha, Wentao Zhu, Tingxun Lv +2
Spatio-temporal representational learning has been widely adopted in various fields such as action recognition, video object segmentation, and action anticipation. Previous spatio-…
Hand Image Understanding via Deep Multi-Task Learning
Xiong Zhang, Hongsheng Huang, Jianchao Tan +5
Analyzing and understanding hand information from multimedia materials like images or videos is important for many real world applications and remains active in research community.…
BAGUA: Scaling up Distributed Learning with System Relaxations
Shaoduo Gan, Xiangru Lian, Rui Wang +11
Recent years have witnessed a growing list of systems for distributed data-parallel training. Existing systems largely fit into two paradigms, i.e., parameter server and MPI-style…
DouZero: Mastering DouDizhu with Self-Play Deep Reinforcement Learning
Daochen Zha, Jingru Xie, Wenye Ma +4
Games are abstractions of the real world, where artificial agents learn to compete and cooperate with other agents. While significant achievements have been made in various perfect…
Distributed Learning Systems with First-order Methods
Ji Liu, Ce Zhang
Scalable and efficient distributed learning is one of the main driving forces behind the recent rapid advancement of machine learning and artificial intelligence. One prominent fea…
From Distributed Machine Learning to Federated Learning: A Survey
Ji Liu, Jizhou Huang, Yang Zhou +4
In recent years, data and computing resources are typically distributed in the devices of end users, various regions or organizations. Because of laws or regulations, the distribut…
Rank the Episodes: A Simple Approach for Exploration in Procedurally-Generated Environments
Daochen Zha, Wenye Ma, Lei Yuan +2
Exploration under sparse reward is a long-standing challenge of model-free reinforcement learning. The state-of-the-art methods address this challenge by introducing intrinsic rewa…
1-bit Adam: Communication Efficient Large-Scale Training with Adam's Convergence Speed
Hanlin Tang, Shaoduo Gan, Ammar Ahmad Awan +6
Scalable training of large models (like BERT and GPT-3) requires careful optimization rooted in model design, architecture, and system capabilities. From a system standpoint, commu…
Once-for-All Adversarial Training: In-Situ Tradeoff between Robustness and Accuracy for Free
Haotao Wang, Tianlong Chen, Shupeng Gui +3
Adversarial training and its many variants substantially improve deep network robustness, yet at the cost of compromising standard accuracy. Moreover, the training process is heavy…
Short Video-based Advertisements Evaluation System: Self-Organizing Learning Approach
Yunjie Zhang, Fei Tao, Xudong Liu +6
With the rising of short video apps, such as TikTok, Snapchat and Kwai, advertisement in short-term user-generated videos (UGVs) has become a trending form of advertising. Predicti…
Themes Informed Audio-visual Correspondence Learning
Runze Su, Fei Tao, Xudong Liu +6
The applications of short-term user-generated video (UGV), such as Snapchat, and Youtube short-term videos, booms recently, raising lots of multimodal machine learning tasks. Among…
Ensemble Chinese End-to-End Spoken Language Understanding for Abnormal Event Detection from audio stream
Haoran Wei, Fei Tao, Runze Su +2
Conventional spoken language understanding (SLU) consist of two stages, the first stage maps speech to text by automatic speech recognition (ASR), and the second stage maps text to…