14 citations · 19 across the 14 of their papers we have counts for
14 papers
Stable FP4 Training via Transposition-Invariant Block Quantization
Mehdi Rahimifar, Amin Darabi, Mehran Taghian Jazi +6
Reducing training precision is a key lever for improving the e ciency of large language model (LLM) training, but pushing beyond FP8 to 4-bit oating point (FP4) remains challenging…
MEPA: Multi-Scale Representation Alignment for Visual Autoregressive Modeling with Mixture of Experts
Nuoyan Zhou, Zhijun Tu, Lei Yu +4
Visual AutoRegressive modeling (VAR) has pioneered a coarse-to-fine multi-scale autoregressive generative paradigm, demonstrating strong capabilities in image generation. However,…
ElasticDiT: Efficient Diffusion Transformers via Elastic Architecture and Sparse Attention for High-Resolution Image Generation on Mobile Devices
Kunpeng Du, Haizhen Xie, Sen Lu +11
The Diffusion Transformer (DiT) architecture is the state-of-the-art paradigm for high-fidelity image generation, underpinning models like Stable Diffusion-3 and FLUX.1. However, d…
Mixture of Ranks with Degradation-Aware Routing for One-Step Real-World Image Super-Resolution
Xiao He, Zhijun Tu, Kun Cheng +4
The demonstrated success of sparsely-gated Mixture-of-Experts (MoE) architectures, exemplified by models such as DeepSeek and Grok, has motivated researchers to investigate their a…
Rethinking 1-bit Optimization Leveraging Pre-trained Large Language Models
Zhijun Tu, Jian Li, Yuanyuan Xi +5
1-bit LLM quantization offers significant advantages in reducing storage and computational costs. However, existing methods typically train 1-bit LLMs from scratch, failing to full…
One-Step Diffusion-based Real-World Image Super-Resolution with Visual Perception Distillation
Xue Wu, Jingwei Xin, Zhijun Tu +4
Diffusion-based models have been widely used in various visual generation tasks, showing promising results in image super-resolution (SR), while typically being limited by dozens o…
Autoregressive Image Generation with Vision Full-view Prompt
Miaomiao Cai, Guanjie Wang, Wei Li +4
In autoregressive (AR) image generation, models based on the 'next-token prediction' paradigm of LLMs have shown comparable performance to diffusion models by reducing inductive bi…
Effective Diffusion Transformer Architecture for Image Super-Resolution
Kun Cheng, Lei Yu, Zhijun Tu +7
Recent advances indicate that diffusion models hold great promise in image super-resolution. While the latest methods are primarily based on latent diffusion models with convolutio…
One Step Diffusion-based Super-Resolution with Time-Aware Distillation
Xiao He, Huaao Tang, Zhijun Tu +8
Diffusion-based image super-resolution (SR) methods have shown promise in reconstructing high-resolution images with fine details from low-resolution counterparts. However, these a…
U-DiTs: Downsample Tokens in U-Shaped Diffusion Transformers
Yuchuan Tian, Zhijun Tu, Hanting Chen +3
Diffusion Transformers (DiTs) introduce the transformer architecture to diffusion tasks for latent-space image generation. With an isotropic architecture that chains a series of tr…
LIPT: Latency-aware Image Processing Transformer
Junbo Qiao, Wei Li, Haizhen Xie +5
Transformer is leading a trend in the field of image processing. Despite the great success that existing lightweight image processing transformers have achieved, they are tailored…
IPT-V2: Efficient Image Processing Transformer using Hierarchical Attentions
Zhijun Tu, Kunpeng Du, Hanting Chen +4
Recent advances have demonstrated the powerful capability of transformer architecture in image restoration. However, our analysis indicates that existing transformerbased methods c…
A Survey on Transformer Compression
Yehui Tang, Yunhe Wang, Jianyuan Guo +4
Transformer plays a vital role in the realms of natural language processing (NLP) and computer vision (CV), specially for constructing large language models (LLM) and large vision…
CBQ: Cross-Block Quantization for Large Language Models
Xin Ding, Xiaoyu Liu, Zhijun Tu +8
Post-training quantization (PTQ) has played a key role in compressing large language models (LLMs) with ultra-low costs. However, existing PTQ methods only focus on handling the ou…