activity
20182026
most citedTaming Sparsely Activated Transformer with Stochastic Experts

27 citations · 113 across the 27 of their papers we have counts for

collaborators
Showing cs.LGShow all

9 papers · 1 filter

cs.LG2023

SMURF-THP: Score Matching-based UnceRtainty quantiFication for Transformer Hawkes Process

Zichong Li, Yanbo Xu, Simiao Zuo +4

Transformer Hawkes process models have shown to be successful in modeling event sequence data. However, most of the existing training methods rely on maximizing the likelihood of e…

cs.LG2023★ 7 cited

Robust Multi-Agent Reinforcement Learning via Adversarial Regularization: Theoretical Foundation and Stable Algorithms

Alexander Bukharin, Yan Li, Yue Yu +6

Multi-Agent Reinforcement Learning (MARL) has shown promising results across several domains. Despite this promise, MARL policies often lack robustness and are therefore sensitive…

cs.LG2022

DiP-GNN: Discriminative Pre-Training of Graph Neural Networks

Simiao Zuo, Haoming Jiang, Qingyu Yin +3

Graph neural network (GNN) pre-training methods have been proposed to enhance the power of GNNs. Specifically, a GNN is first pre-trained on a large-scale unlabeled graph and then…

cs.LG2022

Differentially Private Estimation of Hawkes Process

Simiao Zuo, Tianyi Liu, Tuo Zhao +1

Point process models are of great importance in real world applications. In certain critical applications, estimation of point process models involves large amounts of sensitive pe…

cs.LG2022★ 11 cited

PLATON: Pruning Large Transformer Models with Upper Confidence Bound of Weight Importance

Qingru Zhang, Simiao Zuo, Chen Liang +4

Large Transformer-based models have exhibited superior performance in various natural language processing and computer vision tasks. However, these models contain enormous amounts…

cs.LG2021★ 1 cited

Super Tickets in Pre-Trained Language Models: From Model Compression to Improving Generalization

Chen Liang, Simiao Zuo, Minshuo Chen +5

The Lottery Ticket Hypothesis suggests that an over-parametrized network consists of ``lottery tickets'', and training a certain collection of them (i.e., a subnetwork) can match t…

cs.LG2021

Adversarial Regularization as Stackelberg Game: An Unrolled Optimization Approach

Simiao Zuo, Chen Liang, Haoming Jiang +5

Adversarial regularization has been shown to improve the generalization performance of deep learning models in various natural language processing tasks. Existing works usually for…

cs.LG2020

A Hypergradient Approach to Robust Regression without Correspondence

Yujia Xie, Yixiu Mao, Simiao Zuo +4

We consider a variant of regression problem, where the correspondence between input and output data is not available. Such shuffled data is commonly observed in many real world pro…

cs.LG2020★ 22 cited

Transformer Hawkes Process

Simiao Zuo, Haoming Jiang, Zichong Li +2

Modern data acquisition routinely produce massive amounts of event sequence data in various domains, such as social media, healthcare, and financial markets. These data often exhib…