661 citations · 1.6k across the 26 of their papers we have counts for
26 papers
Towards Automated Model Design on Recommender Systems
Tunhou Zhang, Dehua Cheng, Yuchen He +10
The increasing popularity of deep learning models has created new opportunities for developing AI-based recommender systems. Designing recommender systems using deep neural network…
DistDNAS: Search Efficient Feature Interactions within 2 Hours
Tunhou Zhang, Wei Wen, Igor Fedorov +8
Search efficiency and serving efficiency are two major axes in building feature interactions and expediting the model development process in recommender systems. On large-scale ben…
Farthest Greedy Path Sampling for Two-shot Recommender Search
Yufan Cao, Tunhou Zhang, Wei Wen +3
Weight-sharing Neural Architecture Search (WS-NAS) provides an efficient mechanism for developing end-to-end deep recommender models. However, in complex search spaces, distinguish…
NASRec: Weight Sharing Neural Architecture Search for Recommender Systems
Tunhou Zhang, Dehua Cheng, Yuchen He +7
The rise of deep neural networks offers new opportunities in optimizing recommender systems. However, optimizing recommender systems using deep neural networks requires delicate ar…
TRP: Trained Rank Pruning for Efficient Deep Neural Networks
Yuhui Xu, Yuxi Li, Shuai Zhang +6
To enable DNNs on edge devices like mobile phones, low-rank approximation has been widely adopted because of its solid theoretical rationale and efficient implementations. Several…
Learning Low-rank Deep Neural Networks via Singular Vector Orthogonality Regularization and Singular Value Sparsification
Huanrui Yang, Minxue Tang, Wei Wen +5
Modern deep neural networks (DNNs) often require high memory consumption and large computational loads. In order to deploy DNN algorithms efficiently on edge or mobile devices, a s…
Line Art Correlation Matching Feature Transfer Network for Automatic Animation Colorization
Zhang Qian, Wang Bo, Wen Wei +2
Automatic animation line art colorization is a challenging computer vision problem, since the information of the line art is highly sparse and abstracted and there exists a strict…
Neural Predictor for Neural Architecture Search
Wei Wen, Hanxiao Liu, Hai Li +3
Neural Architecture Search methods are effective but often use complex algorithms to come up with the best architecture. We propose an approach with three basic steps that is conce…
Trained Rank Pruning for Efficient Deep Neural Networks
Yuhui Xu, Yuxi Li, Shuai Zhang +7
To accelerate DNNs inference, low-rank approximation has been widely adopted because of its solid theoretical rationale and efficient implementations. Several previous works attemp…
Conditional Transferring Features: Scaling GANs to Thousands of Classes with 30% Less High-quality Data for Training
Chunpeng Wu, Wei Wen, Yiran Chen +1
Generative adversarial network (GAN) has greatly improved the quality of unsupervised image generation. Previous GAN-based methods often require a large amount of high-quality trai…
DeepHoyer: Learning Sparser Neural Network with Differentiable Scale-Invariant Sparsity Measures
Huanrui Yang, Wei Wen, Hai Li
In seeking for sparse and efficient neural network models, many previous works investigated on enforcing L1 or L0 regularizers to encourage weight sparsity during training. The L0…
Joint Regularization on Activations and Weights for Efficient Neural Network Pruning
Qing Yang, Wei Wen, Zuoguan Wang +1
With the rapid scaling up of deep neural networks (DNNs), extensive research studies on network model compression such as weight pruning have been performed for improving deploymen…
AutoGrow: Automatic Layer Growing in Deep Convolutional Networks
Wei Wen, Feng Yan, Yiran Chen +1
Depth is a key component of Deep Neural Networks (DNNs), however, designing depth is heuristic and requires many human efforts. We propose AutoGrow to automate depth discovery in D…
PruneTrain: Fast Neural Network Training by Dynamic Sparse Model Reconfiguration
Sangkug Lym, Esha Choukse, Siavash Zangeneh +3
State-of-the-art convolutional neural networks (CNNs) used in vision applications have large models with numerous weights. Training these models is very compute- and memory-resourc…
Trained Rank Pruning for Efficient Deep Neural Networks
Yuhui Xu, Yuxi Li, Shuai Zhang +6
The performance of Deep Neural Networks (DNNs) keeps elevating in recent years with increasing network depth and width. To enable DNNs on edge devices like mobile phones, researche…
Mini-batch Serialization: CNN Training with Inter-layer Data Reuse
Sangkug Lym, Armand Behroozi, Wei Wen +3
Training convolutional neural networks (CNNs) requires intense computations and high memory bandwidth. We find that bandwidth today is over-provisioned because most memory accesses…
SmoothOut: Smoothing Out Sharp Minima to Improve Generalization in Deep Learning
Wei Wen, Yandan Wang, Feng Yan +4
In Deep Learning, Stochastic Gradient Descent (SGD) is usually selected as a training method because of its efficiency; however, recently, a problem in SGD gains research interest:…
Learning Intrinsic Sparse Structures within Long Short-Term Memory
Wei Wen, Yuxiong He, Samyam Rajbhandari +6
Model compression is significant for the wide adoption of Recurrent Neural Networks (RNNs) in both user devices possessing limited resources and business clusters requiring quick r…
TernGrad: Ternary Gradients to Reduce Communication in Distributed Deep Learning
Wei Wen, Cong Xu, Feng Yan +4
High network communication cost for synchronizing gradients and parameters is the well-known bottleneck of distributed training. In this work, we propose TernGrad that uses ternary…
A Compact DNN: Approaching GoogLeNet-Level Accuracy of Classification and Domain Adaptation
Chunpeng Wu, Wei Wen, Tariq Afzal +3
Recently, DNN model compression based on network architecture design, e.g., SqueezeNet, attracted a lot attention. No accuracy drop on image classification is observed on these ext…
Coordinating Filters for Faster Deep Neural Networks
Wei Wen, Cong Xu, Chunpeng Wu +3
Very large-scale Deep Neural Networks (DNNs) have achieved remarkable successes in a large variety of computer vision tasks. However, the high computation intensity of DNNs makes i…
Group Scissor: Scaling Neuromorphic Computing Design to Large Neural Networks
Yandan Wang, Wei Wen, Beiye Liu +2
Synapse crossbar is an elementary structure in Neuromorphic Computing Systems (NCS). However, the limited size of crossbars and heavy routing congestion impedes the NCS implementat…
Classification Accuracy Improvement for Neuromorphic Computing Systems with One-level Precision Synapses
Yandan Wang, Wei Wen, Linghao Song +1
Brain inspired neuromorphic computing has demonstrated remarkable advantages over traditional von Neumann architecture for its high energy efficiency and parallel data processing.…
Learning Structured Sparsity in Deep Neural Networks
Wei Wen, Chunpeng Wu, Yandan Wang +2
High demand for computation resources severely hinders deployment of large-scale Deep Neural Networks (DNN) in resource constrained devices. In this work, we propose a Structured S…