1 citations · 1 across the 9 of their papers we have counts for
10 papers
AgentWeave: Efficient Distributed Agent Serving via Flow Decomposition
Kaibin Guo, Pengtu Li, Zicong Hong +4
Agent applications are evolving into workflows composed of large language model (LLM) calls and tool invocations, termed modules. Existing distributed agent systems typically adopt…
ProDVI: Programmatic Dynamics Priors for Value Network Initialization
Xinwei Liu, Junyuan Liang, Jianting Zhang +1
Deep Reinforcement Learning (RL) is notoriously sample inefficient. One contributing factor is that RL agents are typically initialized from scratch, forcing them to acquire task-r…
Observation-Grounded Self-Predictive Reinforcement Learning for Visual Continuous Control
Xinwei Liu, Junyuan Liang, Jianting Zhang +1
Sample-efficient policy learning from pixels is a long-standing challenge in reinforcement learning (RL). Recent dynamics-based representation learning methods have significantly i…
Hawk: Harnessing Hardware-Aware Knowledge for High-Performance NPU Kernel Generation
Junyi Wen, Ruiyan Zhuang, Yongjia Xu +7
Developing high-performance kernels for Neural Processing Units (NPUs) is a critical industry bottleneck, requiring developers to manually navigate implicit hardware constraints an…
NASDAQ: Normalized Observation Space Dynamics-Augmented Q-Learning
Xinwei Liu, Junyuan Liang, Zicong Hong +2
Augmenting model-free reinforcement learning (RL) with representations learned through observation dynamics prediction (observation-predictive RL) can improve sample efficiency and…
DyMoE: Dynamic Expert Orchestration with Mixed-Precision Quantization for Efficient MoE Inference on Edge
Yuegui Huang, Zhiyuan Fang, Weiqi Luo +3
Despite the computational efficiency of MoE models, the excessive memory footprint and I/O overhead inherent in multi-expert architectures pose formidable challenges for real-time…
Krul: Efficient State Restoration for Multi-turn Conversations with Dynamic Cross-layer KV Sharing
Junyi Wen, Junyuan Liang, Zicong Hong +3
Efficient state restoration in multi-turn conversations with large language models (LLMs) remains a critical challenge, primarily due to the overhead of recomputing or loading full…
Klotski: Efficient Mixture-of-Expert Inference via Expert-Aware Multi-Batch Pipeline
Zhiyuan Fang, Yuegui Huang, Zicong Hong +5
Mixture of Experts (MoE), with its distinctive sparse structure, enables the scaling of language models up to trillions of parameters without significantly increasing computational…
Fate: Fast Edge Inference of Mixture-of-Experts Models via Cross-Layer Gate
Zhiyuan Fang, Zicong Hong, Yuegui Huang +5
Large Language Models (LLMs) have demonstrated impressive performance across various tasks, and their application in edge scenarios has attracted significant attention. However, sp…
Training and Serving System of Foundation Models: A Comprehensive Survey
Jiahang Zhou, Yanyu Chen, Zicong Hong +6
Foundation models (e.g., ChatGPT, DALL-E, PengCheng Mind, PanGu-) have demonstrated extraordinary performance in key technological areas, such as natural language processing and…