5 papers
M2XFP: A Metadata-Augmented Microscaling Data Format for Efficient Low-bit Quantization
Weiming Hu, Zihan Zhang, Haoyan Zhang +8
Existing low-bit Microscaling (MX) formats, such as MXFP4, often suffer from substantial accuracy degradation due to the use of a shared scaling factor with the Power-of-Two format…
Sandwich Reasoning: An Answer-Reasoning-Answer Approach for Low-Latency Query Correction
Chen Zhang, Kepu Zhang, Jiatong Zhang +2
Query correction is a critical entry point in modern search pipelines, demanding high accuracy strictly within real-time latency constraints. Chain-of-Thought (CoT) reasoning impro…
FlashFuser: Expanding the Scale of Kernel Fusion for Compute-Intensive Operators via Inter-Core Connection
Ziyu Huang, Yangjie Zhou, Zihan Liu +8
The scaling of computation throughput continues to outpace improvements in memory bandwidth, making many deep learning workloads memory-bound. Kernel fusion is a key technique to a…
ClusterFusion: Expanding Operator Fusion Scope for LLM Inference via Cluster-Level Collective Primitive
Xinhao Luo, Zihan Liu, Yangjie Zhou +8
Large language model (LLM) decoding suffers from high latency due to fragmented execution across operators and heavy reliance on off-chip memory for data exchange and reduction. Th…
Mathematical Computation on High-dimensional Data via Array Programming and Parallel Acceleration
Chen Zhang
While deep learning excels in natural image and language processing, its application to high-dimensional data faces computational challenges due to the dimensionality curse. Curren…