4 papers
LatentRefusal: Latent-Signal Refusal for Unanswerable Text-to-SQL Queries
Xuancheng Ren, Shijing Hu, Zhihui Lu +2
In LLM-based text-to-SQL systems, unanswerable and underspecified user queries may generate not only incorrect text but also executable programs that yield misleading results or vi…
Prefix Probing: Lightweight Harmful Content Detection for Large Language Models
Jirui Yang, Hengqi Guo, Zhihui Lu +6
Large language models often face a three-way trade-off among detection accuracy, inference latency, and deployment cost when used in real-world safety-sensitive applications. This…
Bridging Draft Policy Misalignment: Group Tree Optimization for Speculative Decoding
Shijing Hu, Jingyang Li, Zhihui Lu +1
Speculative decoding accelerates large language model (LLM) inference by letting a lightweight draft model propose multiple tokens that the target model verifies in parallel. Yet e…
GRIFFIN: Effective Token Alignment for Faster Speculative Decoding
Shijing Hu, Jingyang Li, Xingyu Xie +3
Speculative decoding accelerates inference in large language models (LLMs) by generating multiple draft tokens simultaneously. However, existing methods often struggle with token m…