6 papers · 1 filter
MoECa: Aligning Feature Reuse with Expert Decomposition in Diffusion Transformers
Maoliang Li, Haojing Chen, Jiayu Chen +4
Diffusion Transformers with Mixture-of-Experts (DiT-MoE) improve model capacity under sparse activation, but diffusion inference is still bottlenecked by redundant computation acro…
DyQ-VLA: Temporal-Dynamic-Aware Quantization for Embodied Vision-Language-Action Models
Zihao Zheng, Hangyu Cao, Sicheng Tian +12
Vision-language-action (VLA) models achieve favorable task performance, yet runtime errors in closed-loop execution evolve with alternating updates of actions and observations. Exi…
Vision Transformers that Never Stop Learning
Caihao Sun, Mingqi Yuan, Shiyuan Wang +1
Loss of plasticity refers to the progressive inability of a model to adapt to new tasks and poses a fundamental challenge for continual learning. While this phenomenon has been ext…
FedHQ: Hybrid Runtime Quantization for Federated Learning
Zihao Zheng, Ziyao Wang, Xiuping Cui +6
Federated Learning (FL) is a decentralized model training approach that preserves data privacy but struggles with low efficiency. Quantization, a powerful training optimization tec…
DynaMo: Runtime Switchable Quantization for MoE with Cross-Dataset Adaptation
Zihao Zheng, Xiuping Cui, Size Zheng +4
As the Mix-of-Experts (MoE) architecture increases the number of parameters in large models, there is an even greater need for model quantization. However, existing quantization me…
Threshold Neuron: A Brain-inspired Artificial Neuron for Efficient On-device Inference
Zihao Zheng, Yuanchun Li, Jiayu Chen +3
Enhancing the computational efficiency of on-device Deep Neural Networks (DNNs) remains a significant challengein mobile and edge computing. As we aim to execute increasingly compl…