4 papers
FastDraft: How to Train Your Draft
Ofir Zafrir, Igor Margulis, Dorin Shteyman +2
Speculative Decoding has gained popularity as an effective technique for accelerating the auto-regressive inference process of Large Language Models. However, Speculative Decoding…
Tradeoffs Between Alignment and Helpfulness in Language Models with Steering Methods
Yotam Wolf, Noam Wies, Dorin Shteyman +3
Language model alignment has become an important component of AI safety, allowing safe interactions between humans and language models, by enhancing desired behaviors and inhibitin…
WHISTRESS: Enriching Transcriptions with Sentence Stress Detection
Iddo Yosha, Dorin Shteyman, Yossi Adi
Spoken language conveys meaning not only through words but also through intonation, emotion, and emphasis. Sentence stress, the emphasis placed on specific words within a sentence,…
Compositional Hardness of Code in Large Language Models -- A Probabilistic Perspective
Yotam Wolf, Binyamin Rothberg, Dorin Shteyman +1
A common practice in large language model (LLM) usage for complex analytical tasks such as code generation, is to sample a solution for the entire task within the model's context w…