5 papers
Efficient Mixture-of-Agents Serving via Tree-Structured Routing, Adaptive Pruning, and Dependency-Aware Prefill-Decode Overlap
Zijun Wang, Yijiahao Qi, Hanqiu Chen +5
Mixture-of-Agents (MoA) inference can suffer from dense inter-agent communication and low hardware utilization, which jointly inflate serving latency. We present a serving design t…
Step-3 is Large yet Affordable: Model-system Co-design for Cost-effective Decoding
StepFun, :, Bin Wang +195
Large language models (LLMs) face low hardware efficiency during decoding, especially for long-context reasoning tasks. This paper introduces Step-3, a 321B-parameter VLM with hard…
ForgeBench: A Machine Learning Benchmark Suite and Auto-Generation Framework for Next-Generation HLS Tools
Andy Wanna, Hanqiu Chen, Cong Hao
Although High-Level Synthesis (HLS) has attracted considerable interest in hardware design, it has not yet become mainstream due to two primary challenges. First, current HLS hardw…
Residual-INR: Communication Efficient On-Device Learning Using Implicit Neural Representation
Hanqiu Chen, Xuebin Yao, Pradeep Subedi +1
Edge computing is a distributed computing paradigm that collects and processes data at or near the source of data generation. The on-device learning at edge relies on device-to-dev…
HLSFactory: A Framework Empowering High-Level Synthesis Datasets for Machine Learning and Beyond
Stefan Abi-Karam, Rishov Sarkar, Allison Seigler +7
Machine learning (ML) techniques have been applied to high-level synthesis (HLS) flows for quality-of-result (QoR) prediction and design space exploration (DSE). Nevertheless, the…