9 papers
RVANNS: Mixed-Precision Indexing and Locality-Aware Graph Traversal on RISC-V
Chengying Huan, Yudong Liu, Jianguo Wang +9
Approximate nearest neighbor search (ANNS) on CPUs is increasingly constrained by candidate-vector movement and decoding rather than peak arithmetic throughput. Although the RISC-V…
Bole: Efficient Tree Speculation for Hybrid-Attention Language Models
Li Wang, Yi Su, Xiabao Wu +9
Hybrid-attention large language models combine full attention with recurrent linear attention to reduce long-context inference costs, yet their autoregressive decoding remains memo…
TIDE-MC: Two-Sided Interpolative Decomposition for Billion-Scale GPU Matrix Completion
Chengying Huan, Yubo Wang, Pinhuan Wang +11
Matrix completion supports large-scale recommendation and scientific computing, yet existing GPU solvers commonly assume that the observed matrix or its dense factors fit in device…
OrchANN: Hierarchical Orchestration for Skewed Out-of-Core Vector Search
Lizheng Chen, Pinhuan Wang, Shaonan Ma +9
At billion scale, approximate nearest neighbor search (ANNS) often becomes an out-of-core problem: the full vector collection and index structures exceed memory capacity, making qu…
STAR: Decode-Phase Rescheduling for LLM Inference
Zhibin Wang, Zetao Hong, Xue Li +8
Large Language Model (LLM) inference has emerged as a fundamental paradigm, however, variations in output length cause severe workload imbalance in the decode phase, particularly f…
GPU Acceleration of TFHE-Based High-Precision Nonlinear Layers for Encrypted LLM Inference
Guoci Chen, Xiurui Pan, Qiao Li +5
Deploying large language models (LLMs) as cloud services raises privacy concerns as inference may leak sensitive data. Fully Homomorphic Encryption (FHE) allows computation on encr…