3 papers
cs.AI2024
SwapMoE: Serving Off-the-shelf MoE-based Large Language Models with Tunable Memory Budget
Rui Kong, Yuanchun Li, Qingtian Feng +5
Mixture of experts (MoE) is a popular technique to improve capacity of Large Language Models (LLMs) with conditionally-activated parallel experts. However, serving MoE models on me…
cs.CV2024
Xformer: Hybrid X-Shaped Transformer for Image Denoising
Jiale Zhang, Yulun Zhang, Jinjin Gu +3
In this paper, we present a hybrid X-shaped vision Transformer, named Xformer, which performs notably on image denoising tasks. We explore strengthening the global representation o…
cs.CV2024
Recursive Generalization Transformer for Image Super-Resolution
Zheng Chen, Yulun Zhang, Jinjin Gu +2
Transformer architectures have exhibited remarkable performance in image super-resolution (SR). Since the quadratic computational complexity of the self-attention (SA) in Transform…