298 citations · 501 across the 17 of their papers we have counts for
18 papers
GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models
5 Team, Aohan Zeng, Xin Lv +167
We present GLM-4.5, an open-source Mixture-of-Experts (MoE) large language model with 355B total parameters and 32B activated parameters, featuring a hybrid reasoning method that s…
HARMONY: A Scalable Distributed Vector Database for High-Throughput Approximate Nearest Neighbor Search
Qian Xu, Feng Zhang, Chengxi Li +4
Approximate Nearest Neighbor Search (ANNS) is essential for various data-intensive applications, including recommendation systems, image retrieval, and machine learning. Scaling AN…
Optimal Kernel Orchestration for Tensor Programs with Korch
Muyan Hu, Ashwin Venkatram, Shreyashri Biswas +7
Kernel orchestration is the task of mapping the computation defined in different operators of a deep neural network (DNN) to the execution of GPU kernels on modern hardware platfor…
PowerFusion: A Tensor Compiler with Explicit Data Movement Description and Instruction-level Graph IR
Zixuan Ma, Haojie Wang, Jingze Xing +7
Deep neural networks (DNNs) are of critical use in different domains. To accelerate DNN computation, tensor compilers are proposed to generate efficient code on different domain-sp…
FreshGNN: Reducing Memory Access via Stable Historical Embeddings for Graph Neural Network Training
Kezhao Huang, Haitian Jiang, Minjie Wang +7
A key performance bottleneck when training graph neural network (GNN) models on large, real-world graphs is loading node features onto a GPU. Due to limited GPU memory, expensive d…
GLM-130B: An Open Bilingual Pre-trained Model
Aohan Zeng, Xiao Liu, Zhengxiao Du +15
We introduce GLM-130B, a bilingual (English and Chinese) pre-trained language model with 130 billion parameters. It is an attempt to open-source a 100B-scale model at least as good…
Unveiling the Black Box of PLMs with Semantic Anchors: Towards Interpretable Neural Semantic Parsing
Lunyiu Nie, Jiuding Sun, Yanlin Wang +6
The recent prevalence of pretrained language models (PLMs) has dramatically shifted the paradigm of semantic parsing, where the mapping from natural language utterances to structur…
OLLIE: Derivation-based Tensor Program Optimizer
Liyan Zheng, Haojie Wang, Jidong Zhai +7
Boosting the runtime performance of deep neural networks (DNNs) is critical due to their wide adoption in real-world tasks. Existing approaches to optimizing the tensor algebra exp…
GraphQ IR: Unifying the Semantic Parsing of Graph Query Languages with One Intermediate Representation
Lunyiu Nie, Shulin Cao, Jiaxin Shi +5
Subject to the huge semantic gap between natural and formal languages, neural semantic parsing is typically bottlenecked by its complexity of dealing with both input semantics and…
A Roadmap for Big Model
Sha Yuan, Hanyu Zhao, Shuai Zhao +97
With the rapid development of deep learning, training Big Models (BMs) for multiple downstream tasks becomes a popular paradigm. Researchers have achieved various outcomes in the c…
Suppressing ZZ Crosstalk of Quantum Computers through Pulse and Scheduling Co-Optimization
Lei Xie, Jidong Zhai, Zhenxing Zhang +3
Noise is a significant obstacle to quantum computing, and crosstalk is one of the most destructive types of noise affecting superconducting qubits. Previous approaches to supp…
G-TADOC: Enabling Efficient GPU-Based Text Analytics without Decompression
Feng Zhang, Zaifeng Pan, Yanliang Zhou +4
Text analytics directly on compression (TADOC) has proven to be a promising technology for big data analytics. GPUs are extremely popular accelerators for data analytics systems. U…
FastMoE: A Fast Mixture-of-Expert Training System
Jiaao He, Jiezhong Qiu, Aohan Zeng +3
Mixture-of-Expert (MoE) presents a strong potential in enlarging the size of language model to trillions of parameters. However, training trillion-scale MoE requires algorithm and…
GraphPi: High Performance Graph Pattern Matching through Effective Redundancy Elimination
Tianhui Shi, Mingshu Zhai, Yi Xu +1
Graph pattern matching, which aims to discover structural patterns in graphs, is considered one of the most fundamental graph mining problems in many real applications. Despite pre…
TADOC: Text Analytics Directly on Compression
Feng Zhang, Jidong Zhai, Xipeng Shen +5
This article provides a comprehensive description of Text Analytics Directly on Compression (TADOC), which enables direct document analytics on compressed textual data. The article…
ScalAna: Automating Scaling Loss Detection with Graph Analysis
Yuyang Jin, Haojie Wang, Teng Yu +4
Scaling a parallel program to modern supercomputers is challenging due to inter-process communication, Amdahl's law, and resource contention. Performance analysis tools for finding…
AIPerf: Automated machine learning as an AI-HPC benchmark
Zhixiang Ren, Yongheng Liu, Tianhui Shi +6
The plethora of complex artificial intelligence (AI) algorithms and available high performance computing (HPC) power stimulates the expeditious development of AI components with he…
Algorithm-Directed Crash Consistence in Non-Volatile Memory for HPC
Shuo Yang, Kai Wu, Yifan Qiao +2
Fault tolerance is one of the major design goals for HPC. The emergence of non-volatile memories (NVM) provides a solution to build fault tolerant HPC. Data in NVM-based main memor…