11 citations · 22 across the 5 of their papers we have counts for
5 papers
Unmasking Transformers: A Theoretical Approach to Data Recovery via Attention Weights
Yichuan Deng, Zhao Song, Shenghao Xie +1
In the realm of deep learning, transformers have emerged as a dominant architecture, particularly in natural language processing tasks. However, with their widespread adoption, con…
PatchBackdoor: Backdoor Attack against Deep Neural Networks without Model Modification
Yizhen Yuan, Rui Kong, Shenghao Xie +2
Backdoor attack is a major threat to deep learning systems in safety-critical scenarios, which aims to trigger misbehavior of neural network models under attacker-controlled condit…
Convergence of Two-Layer Regression with Nonlinear Units
Yichuan Deng, Zhao Song, Shenghao Xie
Large language models (LLMs), such as ChatGPT and GPT4, have shown outstanding performance in many human life task. Attention computation plays an important role in training LLMs.…
In-Context Learning for Attention Scheme: from Single Softmax Regression to Multiple Softmax Regression via a Tensor Trick
Yeqi Gao, Zhao Song, Shenghao Xie
Large language models (LLMs) have brought significant and transformative changes in human society. These models have demonstrated remarkable capabilities in natural language unders…
WM-MoE: Weather-aware Multi-scale Mixture-of-Experts for Blind Adverse Weather Removal
Yulin Luo, Rui Zhao, Xiaobao Wei +7
Adverse weather removal tasks like deraining, desnowing, and dehazing are usually treated as separate tasks. However, in practical autonomous driving scenarios, the type, intensity…