83 citations · 286 across the 16 of their papers we have counts for
17 papers
FFSplit: Split Feed-Forward Network For Optimizing Accuracy-Efficiency Trade-off in Language Model Inference
Zirui Liu, Qingquan Song, Qiang Charles Xiao +4
The large number of parameters in Pretrained Language Models enhance their performance, but also make them resource-intensive, making it challenging to deploy them on commodity har…
QuantEase: Optimization-based Quantization for Language Models
Kayhan Behdin, Ayan Acharya, Aman Gupta +4
With the rising popularity of Large Language Models (LLMs), there has been an increasing interest in compression techniques that enable their efficient deployment. This study focus…
mSAM: Micro-Batch-Averaged Sharpness-Aware Minimization
Kayhan Behdin, Qingquan Song, Aman Gupta +7
Modern deep learning models are over-parameterized, where different optima can result in widely varying generalization performance. The Sharpness-Aware Minimization (SAM) technique…
Improved Deep Neural Network Generalization Using m-Sharpness-Aware Minimization
Kayhan Behdin, Qingquan Song, Aman Gupta +4
Modern deep learning models are over-parameterized, where the optimization setup strongly affects the generalization performance. A key element of reliable optimization for these s…
Geometric Graph Representation Learning via Maximizing Rate Reduction
Xiaotian Han, Zhimeng Jiang, Ninghao Liu +3
Learning discriminative node representations benefits various downstream tasks in graph analysis such as community detection and node classification. Existing graph representation…
Towards Interaction Detection Using Topological Analysis on Neural Networks
Zirui Liu, Qingquan Song, Kaixiong Zhou +3
Detecting statistical interactions between input features is a crucial and challenging task. Recent advances demonstrate that it is possible to extract learned interactions from tr…
Towards Automated Neural Interaction Discovery for Click-Through Rate Prediction
Qingquan Song, Dehua Cheng, Hanning Zhou +3
Click-Through Rate (CTR) prediction is one of the most important machine learning tasks in recommender systems, driving personalized experience for billions of consumers. Neural ar…
AutoRec: An Automated Recommender System
Ting-Hsiang Wang, Qingquan Song, Xiaotian Han +3
Realistic recommender systems are often required to adapt to ever-changing data and tasks or to explore different models systematically. To address the need, we present AutoRec, an…
Multi-Channel Graph Convolutional Networks
Kaixiong Zhou, Qingquan Song, Xiao Huang +3
Graph neural networks (GNN) has been demonstrated to be effective in classifying graph structures. To further improve the graph representation learning ability, hierarchical GNN ha…
Sub-Architecture Ensemble Pruning in Neural Architecture Search
Yijun Bian, Qingquan Song, Mengnan Du +3
Neural architecture search (NAS) is gaining more and more attention in recent years due to its flexibility and remarkable capability to reduce the burden of neural network design.…
Auto-GNN: Neural Architecture Search of Graph Neural Networks
Kaixiong Zhou, Qingquan Song, Xiao Huang +1
Graph neural networks (GNN) has been successfully applied to operate on the graph-structured data. Given a specific scenario, rich human expertise and tremendous laborious trials a…
Techniques for Automated Machine Learning
Yi-Wei Chen, Qingquan Song, Xia Hu
Automated machine learning (AutoML) aims to find optimal machine learning solutions automatically given a machine learning problem. It could release the burden of data scientists f…
Coupled Variational Recurrent Collaborative Filtering
Qingquan Song, Shiyu Chang, Xia Hu
We focus on the problem of streaming recommender system and explore novel collaborative filtering algorithms to handle the data dynamicity and complexity in a streaming manner. Alt…
Multi-Label Adversarial Perturbations
Qingquan Song, Haifeng Jin, Xiao Huang +1
Adversarial examples are delicately perturbed inputs, which aim to mislead machine learning models towards incorrect outputs. While most of the existing work focuses on generating…
Auto-Keras: An Efficient Neural Architecture Search System
Haifeng Jin, Qingquan Song, Xia Hu
Neural architecture search (NAS) has been proposed to automatically tune deep neural networks, but existing search algorithms, e.g., NASNet, PNAS, usually suffer from expensive com…
Towards Explanation of DNN-based Prediction with Guided Feature Inversion
Mengnan Du, Ninghao Liu, Qingquan Song +1
While deep neural networks (DNN) have become an effective computational tool, the prediction results are often criticized by the lack of interpretability, which is essential in man…
Tensor Completion Algorithms in Big Data Analytics
Qingquan Song, Hancheng Ge, James Caverlee +1
Tensor completion is a problem of filling the missing or unobserved entries of partially observed tensors. Due to the multidimensional character of tensors in describing complex da…