9 papers
Learning Latent Protein Languages for Autoregressive Generation
Mahdi Pourmirzaei, Farzaneh Esmaili, Amir Ziashahabi +2
Autoregressive transformers remain comparatively weak for protein sequence and structure generation. We study the role of target representation: amino acid tokens encode residue id…
ASPIRE: Asynchronous Batched Self-Speculative Decoding for Long-Context LLM Inference
Amir Ziashahabi, Hossein Entezari Zarch, Lei Gao +2
Long-context LLM inference is bottlenecked by attention, whose repeated KV-cache reads make decoding memory-bound. Self-speculative decoding alleviates this by drafting tokens with…
Spend Bits Where Queries Look: KV Cache Vector Quantization with Attention-Preserving Transforms
Samuel Fernández-Menduiña, Amir Ziashahabi, Eduardo Pavez +2
Long-context LLM decoding reads the key-value (KV) cache at every step. Loading it takes longer than computing attention over it, so throughput is bandwidth-bound. Hence, reducing…
ORBIT: Training-Free Multi-Attribute Behavioral Steering via Orthogonal Subspace Rotation
Narges Ghasemi, Amir Ziashahabi, Salman Avestimehr +1
Language models are widely used in assistant settings, where controlling behavioral attributes is often essential. Activation steering modifies hidden-state representations at infe…
GeoToken: Hierarchical Geolocalization of Images via Next Token Prediction
Narges Ghasemi, Amir Ziashahabi, Salman Avestimehr +1
Image geolocalization, the task of determining an image's geographic origin, poses significant challenges, largely due to visual similarities across disparate locations and the lar…
Reject Only Critical Tokens: Pivot-Aware Speculative Decoding
Amir Ziashahabi, Yavuz Faruk Bakman, Duygu Nur Yaldiz +3
Speculative Decoding (SD) ensures that the output matches the target model's distribution exactly. However, we argue that this distribution matching requirement is too stringent an…
OSMGen: Highly Controllable Satellite Image Synthesis using OpenStreetMap Data
Amir Ziashahabi, Narges Ghasemi, Sajjad Shahabi +3
Accurate and up-to-date geospatial data are essential for urban planning, infrastructure monitoring, and environmental management. Yet, automating urban monitoring remains difficul…
MobiZO: Enabling Efficient LLM Fine-Tuning at the Edge via Inference Engines
Lei Gao, Amir Ziashahabi, Yue Niu +2
Large Language Models (LLMs) are currently pre-trained and fine-tuned on large cloud servers. The next frontier is LLM personalization, where a foundation model can be fine-tuned w…
Renormalization Group flow, Optimal Transport and Diffusion-based Generative Model
Artan Sheshmani, Yi-Zhuang You, Baturalp Buyukates +2
Diffusion-based generative models represent a forefront direction in generative AI research today. Recent studies in physics have suggested that the renormalization group (RG) can…