7 papers
CoCoScale: Leveraging Layer-wise Scaling to Unlock the Potential of Online LLM Serving
Jingfeng Wu, Yiyuan He, Minxian Xu +7
Online large language model (LLM) serving has become the backbone of modern AI applications, powering diverse downstream services through shared hardware clusters. However, modern…
Flash-Searcher: Fast and Effective Web Agents via DAG-Based Parallel Execution
Tianrui Qin, Qianben Chen, Sinuo Wang +10
Large language models (LLMs) have demonstrated remarkable capabilities in complex reasoning tasks when equipped with external tools. However, current frameworks predominantly rely…
BridgeNet: A Unified Multimodal Framework for Bridging 2D and 3D Industrial Anomaly Detection
An Xiang, Zixuan Huang, Xitong Gao +2
Industrial anomaly detection for 2D objects has gained significant attention and achieved progress in anomaly detection (AD) methods. However, identifying 3D depth anomalies using…
Unlock the Potential of Fine-grained LLM Serving via Dynamic Module Scaling
Jingfeng Wu, Yiyuan He, Minxian Xu +3
The rise of large language models (LLMs) has created new opportunities across various fields but has also introduced significant challenges in resource management. Current LLM serv…
OAgents: An Empirical Study of Building Effective Agents
He Zhu, Tianrui Qin, King Zhu +21
Recently, Agentic AI has become an increasingly popular research field. However, we argue that current agent research practices lack standardization and scientific rigor, making it…
Mixture of Weight-shared Heterogeneous Group Attention Experts for Dynamic Token-wise KV Optimization
Guanghui Song, Dongping Liao, Yiren Zhao +3
Transformer models face scalability challenges in causal language modeling (CLM) due to inefficient memory allocation for growing key-value (KV) caches, which strains compute and s…