5 papers
RhinoVLA Technical Report
Huixi Technology, :, Chen Zhang +13
Vision-Language-Action (VLA) models have shown strong potential for robotic manipulation, but real-time deployment on edge hardware remains challenging. In this work, we identify V…
Mixture of Distributions Matters: Dynamic Sparse Attention for Efficient Video Diffusion Transformers
Yuxi Liu, Yipeng Hu, Zekun Zhang +2
While Diffusion Transformers (DiTs) have achieved notable progress in video generation, this long-sequence generation task remains constrained by the quadratic complexity inherent…
RoPeSLR: 3D RoPE-driven Sparse-LowRank Attention for Efficient Diffusion Transformers
Yuxi Liu, Zekun Zhang, Yixiang Cai +3
Diffusion Transformers (DiTs) have revolutionized high-fidelity video generation, yet their attention complexity poses a formidable bottleneck for long-sequence…
CR-Net: Scaling Parameter-Efficient Training with Cross-Layer Low-Rank Structure
Boao Kong, Junzhu Liang, Yuxi Liu +2
Low-rank architectures have become increasingly important for efficient large language model (LLM) pre-training, providing substantial reductions in both parameter complexity and m…
MISA: Memory-Efficient LLMs Optimization with Module-wise Importance Sampling
Yuxi Liu, Renjia Deng, Yutong He +3
The substantial memory demands of pre-training and fine-tuning large language models (LLMs) require memory-efficient optimization algorithms. One promising approach is layer-wise o…