11 citations · 25 across the 30 of their papers we have counts for
13 papers · 1 filter
TRUST: A Decentralized Framework for Auditing Large Language Model Reasoning
Morris Yu-Chao Huang, Zhen Tan, Mohan Zhang +3
Large Language Models generate complex reasoning chains that reveal their decision-making, yet verifying the faithfulness and harmlessness of these intermediate steps remains a cri…
Leave It to the Experts: Detecting Knowledge Distillation via MoE Expert Signatures
Pingzhi Li, Morris Yu-Chao Huang, Zhen Tan +6
Knowledge Distillation (KD) accelerates training of large language models (LLMs) but poses intellectual property protection and LLM diversity risks. Existing KD detection methods b…
Can GRPO Help LLMs Transcend Their Pretraining Origin?
Kangqi Ni, Zhen Tan, Zijie Liu +2
Reinforcement Learning with Verifiable Rewards (RLVR), primarily driven by the Group Relative Policy Optimization (GRPO) algorithm, is a leading approach for enhancing the reasonin…
AsyncSpade: Efficient Test-Time Scaling with Asynchronous Sparse Decoding
Shuqing Luo, Yilin Guan, Pingzhi Li +2
Test-time scaling (TTS) boosts LLM reasoning via long chain-of-thought (CoT), but the linear KV-cache growth amplifies the memory-bound bottleneck of LLM decoding. Query-aware page…
Huawei Cloud Model-as-a-Service on the CloudMatrix384 SuperPod
Ao Xiao, Bangzheng He, Baoquan Zhang +125
Scaled-out MoE LLMs and scaled-up SuperPods create new systems challenges for production Model-as-a-Service (MaaS), requiring disaggregation, low-latency communication, and decentr…
Occult: Optimizing Collaborative Communication across Experts for Accelerated Parallel MoE Training and Inference
Shuqing Luo, Pingzhi Li, Jie Peng +7
Mixture-of-experts (MoE) architectures could achieve impressive computational efficiency with expert parallelism, which relies heavily on all-to-all communication across devices. U…
DOGe: Defensive Output Generation for LLM Protection Against Knowledge Distillation
Pingzhi Li, Zhen Tan, Mohan Zhang +3
Large Language Models (LLMs) represent substantial intellectual and economic investments, yet their effectiveness can inadvertently facilitate model imitation via knowledge distill…
Symbolic Analysis of Grover Search Algorithm via Chain-of-Thought Reasoning and Quantum-Native Tokenization
Min Chen, Jinglei Cheng, Pingzhi Li +3
Understanding the high-level conceptual structure of quantum algorithms from their low-level circuit representations is a critical task for verification, debugging, and education.…
Advancing MoE Efficiency: A Collaboration-Constrained Routing (C2R) Strategy for Better Expert Parallelism Design
Mohan Zhang, Pingzhi Li, Jie Peng +2
Mixture-of-Experts (MoE) has successfully scaled up models while maintaining nearly constant computing costs. By employing a gating network to route input tokens, it selectively ac…
Finding Fantastic Experts in MoEs: A Unified Study for Expert Dropping Strategies and Observations
Ajay Jaiswal, Jianyu Wang, Yixiao Li +6
Sparsely activated Mixture-of-Experts (SMoE) has shown promise in scaling up the learning capacity of neural networks. However, vanilla SMoEs have issues such as expert redundancy…
ORAL: Prompting Your Large-Scale LoRAs via Conditional Recurrent Diffusion
Rana Muhammad Shahroz Khan, Dongwen Tang, Pingzhi Li +2
Parameter generation has emerged as a novel paradigm for neural network development, offering an alternative to traditional neural network training by synthesizing high-quality mod…
Make Optimization Once and for All with Fine-grained Guidance
Mingjia Shi, Ruihan Lin, Xuxi Chen +8
Learning to Optimize (L2O) enhances optimization efficiency with integrated neural networks. L2O paradigms achieve great outcomes, e.g., refitting optimizer, generating unseen solu…
GroverGPT: A Large Language Model with 8 Billion Parameters for Quantum Searching
Haoran Wang, Pingzhi Li, Min Chen +3
Quantum computing is an exciting non-Von Neumann paradigm, offering provable speedups over classical computing for specific problems. However, the practical limits of classical sim…