2 citations · 3 across the 7 of their papers we have counts for
8 papers
RoofLang: Enabling AI-Driven Architecting of LLM Inference Systems
Ziyue Yang, Yuting Jiang, Lei Qu +1
AI is beginning to make substantive contributions to LLM inference optimization. Existing AI optimizations are predominantly profiling-based. Profiling-bound feedback confines the…
Sigma-MoE-Tiny Technical Report
Qingguo Hu, Zhenghao Lin, Ziyue Yang +12
Mixture-of-Experts (MoE) has emerged as a promising paradigm for foundation models due to its efficient and powerful scalability. In this work, we present Sigma-MoE-Tiny, an MoE la…
SIGMA: An AI-Empowered Training Stack on Early-Life Hardware
Lei Qu, Lianhai Ren, Peng Cheng +12
An increasing variety of AI accelerators is being considered for large-scale training. However, enabling large-scale training on early-life AI accelerators faces three core challen…
FengHuang: Next-Generation Memory Orchestration for AI Inferencing
Jiamin Li, Lei Qu, Tao Zhang +4
This document presents a vision for a novel AI infrastructure design that has been initially validated through inference simulations on state-of-the-art large language models. Adva…
Sigma: Differential Rescaling of Query, Key and Value for Efficient Language Models
Zhenghao Lin, Zihao Tang, Xiao Liu +31
We introduce Sigma, an efficient large language model specialized for the system domain, empowered by a novel architecture including DiffQKV attention, and pre-trained on our metic…
NeoMem: Hardware/Software Co-Design for CXL-Native Memory Tiering
Zhe Zhou, Yiqi Chen, Tao Zhang +8
The Compute Express Link (CXL) interconnect makes it feasible to integrate diverse types of memory into servers via its byte-addressable SerDes links. Considering the various acces…
SuperBench: Improving Cloud AI Infrastructure Reliability with Proactive Validation
Yifan Xiong, Yuting Jiang, Ziyue Yang +17
Reliability in cloud AI infrastructure is crucial for cloud service providers, prompting the widespread use of hardware redundancies. However, these redundancies can inadvertently…
CrossoverScheduler: Overlapping Multiple Distributed Training Applications in a Crossover Manner
Cheng Luo, Lei Qu, Youshan Miao +2
Distributed deep learning workloads include throughput-intensive training tasks on the GPU clusters, where the Distributed Stochastic Gradient Descent (SGD) incurs significant comm…