17 papers
A Replicate-and-Quantize Strategy for Plug-and-Play Load Balancing of Sparse Mixture-of-Experts LLMs
Zijie Liu, Jie Peng, Jinhao Duan +7
The paper proposes a training‑free method that replicates heavily used experts and quantizes less important ones to rebalance workload in sparse mixture‑of‑experts large language m…
Pocket-SLAM: Rendering-Area-Aware Pruning for Memory-Efficient 3DGS-SLAM
Leshu Li, Jie Peng, Yang Zhao
3D Gaussian Splatting (3DGS) has garnered significant attention in Simultaneous Localization and Mapping (SLAM) due to its advances in capturing fine-grained geometry features and…
Graph-of-Agents: A Graph-based Framework for Multi-Agent LLM Collaboration
Sukwon Yun, Jie Peng, Pingzhi Li +5
With an ever-growing zoo of LLMs and benchmarks, the need to orchestrate multiple models for improved task performance has never been more pressing. While frameworks like Mixture-o…
PRIME: Prototype-Driven Multimodal Pretraining for Cancer Prognosis with Missing Modalities
Kai Yu, Shuang Zhou, Yiran Song +9
Multimodal self-supervised pretraining offers a promising route to cancer prognosis by integrating histopathology whole-slide images, gene expression, and pathology reports, yet mo…
Understanding the Role of Hallucination in Reinforcement Post-Training of Multimodal Reasoning Models
Gengwei Zhang, Jie Peng, Zhen Tan +6
The recent success of reinforcement learning (RL) in large reasoning models has inspired the growing adoption of RL for post-training Multimodal Large Language Models (MLLMs) to en…
Mozart: Modularized and Efficient MoE Training on 3.5D Wafer-Scale Chiplet Architectures
Shuqing Luo, Ye Han, Pingzhi Li +7
Mixture-of-Experts (MoE) architecture offers enhanced efficiency for Large Language Models (LLMs) with modularized computation, yet its inherent sparsity poses significant hardware…