39 citations · 239 across the 55 of their papers we have counts for
14 papers · 1 filter
NI Sampling: Accelerating Discrete Diffusion Sampling by Token Order Optimization
Enshu Liu, Xuefei Ning, Yu Wang +1
Discrete diffusion language models (dLLMs) have recently emerged as a promising alternative to traditional autoregressive approaches, offering the flexibility to generate tokens in…
Distilled Decoding 2: One-step Sampling of Image Auto-regressive Models with Conditional Score Distillation
Enshu Liu, Qian Chen, Xuefei Ning +4
Image Auto-regressive (AR) models have emerged as a powerful paradigm of visual generative models. Despite their promising performance, they suffer from slow generation speed due t…
Latent Zoning Network: A Unified Principle for Generative Modeling, Representation Learning, and Classification
Zinan Lin, Enshu Liu, Xuefei Ning +3
Generative modeling, representation learning, and classification are three core problems in machine learning (ML), yet their state-of-the-art (SoTA) solutions remain largely disjoi…
Towards Accurate and Efficient Sub-8-Bit Integer Training
Wenjin Guo, Donglai Liu, Weiying Xie +7
Neural network training is a memory- and compute-intensive task. Quantization, which enables low-bitwidth formats in training, can significantly mitigate the workload. To reduce qu…
CSKV: Training-Efficient Channel Shrinking for KV Cache in Long-Context Scenarios
Luning Wang, Shiyao Li, Xuefei Ning +4
Large Language Models (LLMs) have been widely adopted to process long-context tasks. However, the large memory overhead of the key-value (KV) cache poses significant challenges in…
Efficient Expert Pruning for Sparse Mixture-of-Experts Language Models: Enhancing Performance and Reducing Inference Costs
Enshu Liu, Junyi Zhu, Zinan Lin +6
The rapid advancement of large language models (LLMs) has led to architectures with billions to trillions of parameters, posing significant deployment challenges due to their subst…
Mixture of Attention Spans: Optimizing LLM Inference Efficiency with Heterogeneous Sliding-Window Lengths
Tianyu Fu, Haofeng Huang, Xuefei Ning +10
Sliding-window attention offers a hardware-efficient solution to the memory and throughput challenges of Large Language Models (LLMs) in long-context scenarios. Existing methods ty…
OMS-DPM: Optimizing the Model Schedule for Diffusion Probabilistic Models
Enshu Liu, Xuefei Ning, Zinan Lin +2
Diffusion probabilistic models (DPMs) are a new class of generative models that have achieved state-of-the-art generation quality in various domains. Despite the promise, one major…
Dynamic Ensemble of Low-fidelity Experts: Mitigating NAS "Cold-Start"
Junbo Zhao, Xuefei Ning, Enshu Liu +7
Predictor-based Neural Architecture Search (NAS) employs an architecture performance predictor to improve the sample efficiency. However, predictor-based NAS suffers from the sever…
BoolNet: Minimizing The Energy Consumption of Binary Neural Networks
Nianhui Guo, Joseph Bethge, Haojin Yang +4
Recent works on Binary Neural Networks (BNNs) have made promising progress in narrowing the accuracy gap of BNNs to their 32-bit counterparts. However, the accuracy gains are often…
Ensemble-in-One: Learning Ensemble within Random Gated Networks for Enhanced Adversarial Robustness
Yi Cai, Xuefei Ning, Huazhong Yang +1
Adversarial attacks have rendered high security risks on modern deep learning systems. Adversarial training can significantly enhance the robustness of neural network models by sup…
FedCor: Correlation-Based Active Client Selection Strategy for Heterogeneous Federated Learning
Minxue Tang, Xuefei Ning, Yitu Wang +4
Client-wise data heterogeneity is one of the major issues that hinder effective training in federated learning (FL). Since the data distribution on each client may vary dramaticall…
Exploring the Potential of Low-bit Training of Convolutional Neural Networks
Kai Zhong, Xuefei Ning, Guohao Dai +5
In this work, we propose a low-bit training framework for convolutional neural networks, which is built around a novel multi-level scaling (MLS) tensor format. Our framework focuse…
A Generic Graph-based Neural Architecture Encoding Scheme for Predictor-based NAS
Xuefei Ning, Yin Zheng, Tianchen Zhao +2
This work proposes a novel Graph-based neural ArchiTecture Encoding Scheme, a.k.a. GATES, to improve the predictor-based neural architecture search. Specifically, different from ex…