5 papers · 1 filter
Learning Latent Protein Languages for Autoregressive Generation
Mahdi Pourmirzaei, Farzaneh Esmaili, Amir Ziashahabi +2
Autoregressive transformers remain comparatively weak for protein sequence and structure generation. We study the role of target representation: amino acid tokens encode residue id…
ASPIRE: Asynchronous Batched Self-Speculative Decoding for Long-Context LLM Inference
Amir Ziashahabi, Hossein Entezari Zarch, Lei Gao +2
Long-context LLM inference is bottlenecked by attention, whose repeated KV-cache reads make decoding memory-bound. Self-speculative decoding alleviates this by drafting tokens with…
Spend Bits Where Queries Look: KV Cache Vector Quantization with Attention-Preserving Transforms
Samuel Fernández-Menduiña, Amir Ziashahabi, Eduardo Pavez +2
Long-context LLM decoding reads the key-value (KV) cache at every step. Loading it takes longer than computing attention over it, so throughput is bandwidth-bound. Hence, reducing…
Reject Only Critical Tokens: Pivot-Aware Speculative Decoding
Amir Ziashahabi, Yavuz Faruk Bakman, Duygu Nur Yaldiz +3
Speculative Decoding (SD) ensures that the output matches the target model's distribution exactly. However, we argue that this distribution matching requirement is too stringent an…
MobiZO: Enabling Efficient LLM Fine-Tuning at the Edge via Inference Engines
Lei Gao, Amir Ziashahabi, Yue Niu +2
Large Language Models (LLMs) are currently pre-trained and fine-tuned on large cloud servers. The next frontier is LLM personalization, where a foundation model can be fine-tuned w…