16 citations · 20 across the 21 of their papers we have counts for
8 papers · 1 filter
MetaMoE: Diversity-Aware Proxy Selection for Privacy-Preserving Mixture-of-Experts Unification
Weisen Jiang, Shuhao Chen, Sinno Jialin Pan
Mixture-of-Experts (MoE) models scale capacity by combining specialized experts, but most existing approaches assume centralized access to training data. In practice, data are dist…
MetaDefense: Defending Finetuning-based Jailbreak Attack Before and During Generation
Weisen Jiang, Sinno Jialin Pan
This paper introduces MetaDefense, a novel framework for defending against finetuning-based jailbreak attacks in large language models (LLMs). We observe that existing defense mech…
PreMoE: Proactive Inference for Efficient Mixture-of-Experts
Zehua Pei, Ying Zhang, Hui-Ling Zhen +6
Mixture-of-Experts (MoE) models offer dynamic computation, but are typically deployed as static full-capacity models, missing opportunities for deployment-specific specialization.…
KVTuner: Sensitivity-Aware Layer-Wise Mixed-Precision KV Cache Quantization for Efficient and Nearly Lossless LLM Inference
Xing Li, Zeyu Xing, Yiming Li +6
KV cache quantization can improve Large Language Models (LLMs) inference throughput and latency in long contexts and large batch-size scenarios while preserving LLMs effectiveness.…
Analytical FFN-to-MoE Restructuring via Activation Pattern Analysis
Zehua Pei, Hui-Ling Zhen, Lancheng Zou +5
Scaling large language models (LLMs) improves performance but significantly increases inference costs, with feed-forward networks (FFNs) consuming the majority of computational res…
State Chrono Representation for Enhancing Generalization in Reinforcement Learning
Jianda Chen, Wen Zheng Terence Ng, Zichen Chen +2
In reinforcement learning with image-based inputs, it is crucial to establish a robust and generalizable state representation. Recent advancements in metric learning, such as deep…
From Pruning to Grafting: Dynamic Knowledge Redistribution via Learnable Layer Fusion
Zehua Pei, Hui-Ling Zhen, Xianzhi Yu +3
Structured pruning of Generative Pre-trained Transformers (GPTs) offers a promising path to efficiency but often suffers from irreversible performance degradation due to the discar…
Overcoming Negative Transfer by Online Selection: Distant Domain Adaptation for Fault Diagnosis
Ziyan Wang, Mohamed Ragab, Wenmian Yang +4
Unsupervised domain adaptation (UDA) has achieved remarkable success in fault diagnosis, bringing significant benefits to diverse industrial applications. While most UDA methods fo…