4 papers
Contextual Correction of Parallel Drafts for Speculative Decoding
Roy Eisenstadt, Harel Lidar, Tal Gonen +5
Speculative decoding accelerates large language model inference by using a lightweight drafter to propose multiple tokens that are subsequently verified in parallel by the target m…
To Copy or Not to Copy: Controlling Speculative Decoding via Intrinsic Model Signals
Roy Eisenstadt, Ido Cohen, Edo Cohen-Karlik +2
Speculative Decoding (SD) has significantly accelerated Large Language Model (LLM) inference, yet existing approaches face a fundamental tradeoff between two drafting strategies: n…
Overclocking LLM Reasoning: Monitoring and Controlling Thinking Path Lengths in LLMs
Roy Eisenstadt, Itamar Zimerman, Lior Wolf
Recently, techniques such as explicit structured reasoning have demonstrated strong test-time scaling behavior by enforcing a separation between the model's internal "thinking" pro…
An End-to-End Dialogue Summarization System for Sales Calls
Abedelkadir Asi, Song Wang, Roy Eisenstadt +4
Summarizing sales calls is a routine task performed manually by salespeople. We present a production system which combines generative models fine-tuned for customer-agent setting,…