6 papers
HA-RAG: Hotness-Aware RAG Acceleration via Mixed Precision and Data Placement
Danying Ge, Jianhua Gao, Yixue Yang +1
Retrieval-Augmented Generation (RAG) improves model output accuracy by leveraging external knowledge bases, serving as an effective solution to address hallucination issues and kno…
From Profiling to Optimization: Unveiling the Profile Guided Optimization
Bingxin Liu, Yinghui Huang, Jianhua Gao +4
Profile Guided Optimization (PGO) uses runtime profiling to direct compiler optimization decisions, effectively combining static analysis with actual execution behavior to enhance…
Automatic Task Detection and Heterogeneous LLM Speculative Decoding
Danying Ge, Jianhua Gao, Qizhi Jiang +2
Speculative decoding, which combines a draft model with a target model, has emerged as an effective approach to accelerate large language model (LLM) inference. However, existing m…
Cascaded Prediction and Asynchronous Execution of Iterative Algorithms on Heterogeneous Platforms
Jianhua Gao, Bingjie Liu, Yizhuo Wang +2
Owing to the diverse scales and varying distributions of sparse matrices arising from practical problems, a multitude of choices are present in the design and implementation of spa…
Precision-Aware Iterative Algorithms Based on Group-Shared Exponents of Floating-Point Numbers
Jianhua Gao, Jiayuan Shen, Yuxiang Zhang +2
Iterative solvers are frequently used in scientific applications and engineering computations. However, the memory-bound Sparse Matrix-Vector (SpMV) kernel computation hinders the…
Deep Learning-based Software Engineering: Progress, Challenges, and Opportunities
Xiangping Chen, Xing Hu, Yuan Huang +23
Researchers have recently achieved significant advances in deep learning techniques, which in turn has substantially advanced other research disciplines, such as natural language p…