6 papers · 1 filter
AOSpec: Action and Observation Co-Speculation for Low-Latency Agent Serving
Hao Mark Chen, Jinnan Guo, Wayne Luk +1
Large language model agents increasingly act through stateful tools, yet model generation and environment execution remain serialized at every step. As decoding accelerates, tool e…
Dynamic Expert Sharing: Decoupling Memory from Parallelism in Mixture-of-Experts Diffusion LLMs
Hao Mark Chen, Zhiwen Mo, Royson Lee +6
Among parallel decoding paradigms, diffusion large language models (dLLMs) have emerged as a promising candidate that balances generation quality and throughput. However, their int…
FastTTS: Accelerating Test-Time Scaling for Edge LLM Reasoning
Hao Mark Chen, Zhiwen Mo, Guanxi Lu +4
Recent advances in reasoning Large Language Models (LLMs) are driving the emergence of agentic AI systems. Edge deployment of LLM agents near end users is increasingly necessary to…
Enhancing Trustworthiness with Mixed Precision: Benchmarks, Opportunities, and Challenges
Guanxi Lu, Hao Mark Chen, Zhiqiang Que +2
Large language models (LLMs) have shown promising performance across various tasks. However, their autoregressive decoding process poses significant challenges for efficient deploy…
FW-Merging: Scaling Model Merging with Frank-Wolfe Optimization
Hao Mark Chen, Shell Xu Hu, Wayne Luk +2
Model merging has emerged as a promising approach for multi-task learning (MTL), offering a data-efficient alternative to conventional fine-tuning. However, with the rapid developm…
Hardware-Aware Parallel Prompt Decoding for Memory-Efficient Acceleration of LLM Inference
Hao Mark Chen, Wayne Luk, Ka Fai Cedric Yiu +4
The auto-regressive decoding of Large Language Models (LLMs) results in significant overheads in their hardware performance. While recent research has investigated various speculat…