SelfJudge: Faster Speculative Decoding via Self-Supervised Judge Verification

Kanghoon Yoon, Minsub Kim, Sungjae Lee +6

Speculative decoding accelerates LLM inference by verifying candidate tokens from a draft model against a larger target model. Recent judge decoding boosts this process by relaxing…