82 citations · 238 across the 53 of their papers we have counts for
57 papers
Beaver: Elastic GPU Sharing between ML and Latency-Critical vRAN Workloads
Yuncheng Yao, Zhenzhou Qi, Junyao Zheng +3
Within the shared industry vision of AI-RAN, AI-and-RAN seeks to co-locate virtualized radio access network (vRAN) workloads and AI services on shared GPUs. This sharing is inheren…
Vosti: Specifying, Implementing, and Verifying Deterministic LLM Inference
Jianxing Qin, Alexander Du, Danfeng Zhang +2
LLM inference systems may vary batch composition, prompt chunking, prefill/decode execution, and KV-cache reuse, eviction, or recomputation. These optimizations should not affect s…
LowRankArena: A Standardized Evaluation Platform for SVD-Based LLM Compression
Zishan Shao, Lixun Zhang, Kangning Cui +10
SVD-based low-rank compression has become a fast-growing direction for reducing the memory and computational cost of large language models (LLMs). However, meaningful comparison ac…
Iteration Without Elaboration: A Simple ReAct Architecture Suffices for Text-to-SQL Generation
Jian Lu, Haiwei Yu, Raymond M Xiong +2
Modern text-to-SQL systems have become increasingly elaborate, relying on schema-linking modules, retrieval-augmented prompting, candidate generation, and multi-stage refinement pi…
Toward Automated Detection of Documentation Inconsistencies in Electronic Health Records
Jian Lu, Panyu Chen, Miriam Treggiari +5
Objective:To characterize the kinds of internal documentation inconsistencies a general-domain large language model (LLM) can surface from real-world discharge summaries, and to id…
DecodeShare: Tracing the Shared Subspace of LLM Decode-Time Decisions
Zishan Shao, Lixun Zhang, Kangning Cui +10
Large language models (LLMs) handle many tasks with one set of parameters, but under KV-cached inference it is unclear what task-general structure, if any, is used at decode time r…
Optimus: Elastic Decoding for Efficient Diffusion LLM Serving
Chiyue Wei, Cong Guo, Bowen Duan +7
Large language model (LLM) serving is fundamentally limited by inefficient hardware utilization. Autoregressive (AR) decoding underutilizes GPUs due to its strictly sequential exec…
Trellis: Bringing Compiler-Guided Repair into Agentic Code Generation
Alexander Du, Jianjun Ou, Danyang Zhuo +1
Coding agents regularly generate programs that do not compile and require substantial tokens to repair. Yet most repairs are small and local to the edit that caused them. We presen…
FlashSVD v1.5: Making Low-Rank Transformers Inference Actually Fast
Wenhao Wu, Zishan Shao, Kangning Cui +5
SVD-based Low-rank compression reduces transformer parameters and nominal FLOPs, but these savings often translate poorly into real LLM serving speedups. We show that this gap is l…
Parallel Prefix Verification for Speculative Generation
Yuncheng Yao, Yuxuan Xia, Shengjie Wang +1
We introduce PARSE (PArallel pRefix Speculative Engine), a speculative generation framework that accelerates large language model (LLM) inference by parallelizing prefix verificati…
Feature Anchors for Time-Series Sensor-Based Human Activity Recognition
Ruijie Yao, Chenhang Li, Danyang Zhuo +2
Wearable Human Activity Recognition (HAR) still lacks a representation that is both explicit and adaptable. Handcrafted time-series features (TSFs) capture meaningful motion statis…
Foundry: Template-Based CUDA Graph Context Materialization for Fast LLM Serving Cold Start
Xueshen Liu, Yongji Wu, Yuncheng Yao +3
Modern LLM service providers increasingly rely on autoscaling and parallelism reconfiguration to respond to rapidly changing workloads, but cold-start latency remains a major bottl…
Real-Time and Scalable Zak-OTFS Receiver Processing on GPUs
Junyao Zheng, Chung-Hsuan Tung, Yuncheng Yao +6
Orthogonal time frequency space (OTFS) modulation offers superior robustness to high-mobility channels compared to conventional orthogonal frequency-division multiplexing (OFDM) wa…
MARS: Efficient, Adaptive Co-Scheduling for Heterogeneous Agentic Systems
Yifei Wang, Hancheng Ye, Yechen Xu +8
Large language models (LLMs) are increasingly deployed as the execution core of autonomous agents rather than as standalone text generators. Agentic workloads induce a temporal shi…
InfoFlow KV: Information-Flow-Aware KV Recomputation for Long Context
Xin Teng, Canyu Zhang, Shaoyi Zheng +3
Retrieval-augmented generation (RAG) for long-context question answering is bottlenecked by inference-time prefilling over large retrieved contexts. A common strategy is to precomp…
Agentic AI for Scalable and Robust Optical Systems Control
Zehao Wang, Mingzhe Han, Wei Cheng +12
We present AgentOptics, an agentic AI framework for high-fidelity, autonomous optical system control built on the Model Context Protocol (MCP). AgentOptics interprets natural langu…
Nixie: Efficient, Transparent Temporal Multiplexing for Consumer GPUs
Yechen Xu, Yifei Wang, Nathanael Ren +2
Consumer machines are increasingly running large ML workloads such as large language models (LLMs), text-to-image generation, and interactive image editing. Unlike datacenter GPUs,…
Curator: Efficient Vector Search with Low-Selectivity Filters
Yicheng Jin, Yongji Wu, Wenjun Hu +4
Embedding-based dense retrieval has become the cornerstone of many critical applications, where approximate nearest neighbor search (ANNS) queries are often combined with filters o…
RLAX: Large-Scale, Distributed Reinforcement Learning for Large Language Models on TPUs
Runlong Zhou, Lefan Zhang, Shang-Chen Wu +29
Reinforcement learning (RL) has emerged as the de-facto paradigm for improving the reasoning capabilities of large language models (LLMs). We have developed RLAX, a scalable RL fra…
DecodeX: Exploring and Benchmarking of LDPC Decoding across CPU, GPU, and ASIC Platforms
Zhenzhou Qi, Yuncheng Yao, Yiming Li +4
Emerging virtualized radio access networks (vRANs) demand flexible and efficient baseband processing across heterogeneous compute substrates. In this paper, we present DecodeX, a u…
Reliable Curation of EHR Dataset via Large Language Models under Environmental Constraints
Raymond M. Xiong, Panyu Chen, Tianze Dong +6
Electronic health records (EHRs) are central to modern healthcare delivery and research; yet, many researchers lack the database expertise necessary to write complex SQL queries or…
KVCOMM: Online Cross-context KV-cache Communication for Efficient LLM-based Multi-agent Systems
Hancheng Ye, Zhengqi Gao, Mingyuan Ma +8
Multi-agent large language model (LLM) systems are increasingly adopted for complex language processing tasks that require communication and coordination among agents. However, the…
FlashSVD: Memory-Efficient Inference with Streaming for Low-Rank Models
Zishan Shao, Yixiao Wang, Qinsi Wang +6
Singular Value Decomposition (SVD) has recently seen a surge of interest as a simple yet powerful tool for large language models (LLMs) compression, with a growing number of works…
AXLearn: Modular, Hardware-Agnostic Large Model Training
Mark Lee, Chang Lan, Tom Gunter +34
AXLearn is a production system which facilitates scalable and high-performance training of large deep learning models. Compared to other state-of-art deep learning systems, AXLearn…