2 citations · 3 across the 12 of their papers we have counts for
20 papers
The Sharp Directions Are Against You: Curvature Analysis of Activation Steering
Seyedarmin Azizi, Arya Fayyazi, Parsa Razmara +3
Activation steering adds a vector to a language model's internal representations at inference time. It is a lightweight alternative to fine-tuning for behavioral control. The stand…
Running Fewer Loops Removes Refusal Behaviour in Recurrent-Depth Language Models
Seyedarmin Azizi, Erfan Baghaei Potraghloo, Arya Fayyazi +2
A recurrent-depth language model reuses one block of weights several times in a row. The number of repetitions, which we call loops, is chosen when the model runs, not when it is t…
Sharpen Without Search: On-Policy Distillation of Sequence-Level Power Distribution
Erfan Baghaei Potraghloo, Seyedarmin Azizi, Arya Fayyazi +4
A language model can give a correct answer more probability than any single incorrect answer and still usually sample an incorrect one, because the incorrect answers together hold…
Labels Override Definitions in Jev-Style Typed Decision Models
Seyedarmin Azizi, Erfan Baghaei Potraghloo, Massoud Pedram
A typed decision model answers a fixed question about an input by returning a probability for each of several caller-defined options. Each option carries a short label and a writte…
Chopthin-Consensus Power Sampling: A Diversity-Preserving Approach to LLM Decoding
Minoo Ahmadi, Seyedarmin Azizi, Erfan Baghaei Potraghloo +2
Inference-time power sampling via Sequential Monte Carlo (SMC) can substantially improve large language model (LLM) reasoning without requiring post-training. However, many existin…
CircuitSteer: Geometrically Aligned Multi-Layer Steering via Sparse Autoencoder Circuits
Mehrshad Saadatinia, Parsa Razmara, Ardalan Aryashad +2
Controlling the behavior of large language models (LLMs) remains a critical challenge for AI alignment. Existing steering methods, such as Contrastive Activation Addition (CAA), ty…
One Token Away from Collapse: The Fragility of Instruction-Tuned Helpfulness
Erfan Baghaei Potraghloo, Seyedarmin Azizi, Souvik Kundu +1
Instruction-tuned large language models produce helpful, structured responses, but how robust is this helpfulness under trivial constraints? We show that simple lexical constraints…
Power-SMC: Low-Latency Sequence-Level Power Sampling for Training-Free LLM Reasoning
Seyedarmin Azizi, Erfan Baghaei Potraghloo, Minoo Ahmadi +2
Reasoning ability in large language models is often attributed to \emph{distribution sharpening}: concentrating output probability on high-likelihood sequences. Recent works show t…
SkipKV: Selective Skipping of KV Generation and Storage for Efficient Inference with Large Reasoning Models
Jiayi Tian, Seyedarmin Azizi, Yequan Zhao +7
Large reasoning models (LRMs) often incur significant key-value (KV) cache overhead, due to their linear growth with the verbose chain-of-thought (CoT) reasoning. This incurs both…
From Filters to VLMs: Benchmarking Defogging Methods through Object Detection and Segmentation Performance
Ardalan Aryashad, Parsa Razmara, Amin Mahjoub +3
Autonomous driving perception systems are particularly vulnerable in foggy conditions, where light scattering reduces contrast and obscures fine details critical for safe operation…
Top-H Decoding: Adapting the Creativity and Coherence with Bounded Entropy in Text Generation
Erfan Baghaei Potraghloo, Seyedarmin Azizi, Souvik Kundu +1
Large language models (LLMs), despite their impressive performance across a wide range of tasks, often struggle to balance two competing objectives in open-ended text generation: f…
Activation Steering for Chain-of-Thought Compression
Seyedarmin Azizi, Erfan Baghaei Potraghloo, Massoud Pedram
Large language models (LLMs) excel at complex reasoning when they include intermediate steps, known as "chains of thought" (CoTs). However, these rationales are often overly verbos…
VISTA: Vision-Language Inference for Training-Free Stock Time-Series Analysis
Tina Khezresmaeilzadeh, Parsa Razmara, Seyedarmin Azizi +2
Stock price prediction remains a complex and high-stakes task in financial analysis, traditionally addressed using statistical models or, more recently, language models. In this wo…
Efficient Noise Mitigation for Enhancing Inference Accuracy in DNNs on Mixed-Signal Accelerators
Seyedarmin Azizi, Mohammad Erfan Sadeghi, Mehdi Kamal +1
In this paper, we propose a framework to enhance the robustness of the neural models by mitigating the effects of process-induced and aging-related variations of analog computing c…
LaMDA: Large Model Fine-Tuning via Spectrally Decomposed Low-Dimensional Adaptation
Seyedarmin Azizi, Souvik Kundu, Massoud Pedram
Low-rank adaptation (LoRA) has become the default approach to fine-tune large language models (LLMs) due to its significant reduction in trainable parameters. However, trainable pa…
PEANO-ViT: Power-Efficient Approximations of Non-Linearities in Vision Transformers
Mohammad Erfan Sadeghi, Arash Fayyazi, Seyedarmin Azizi +1
The deployment of Vision Transformers (ViTs) on hardware platforms, specially Field-Programmable Gate Arrays (FPGAs), presents many challenges, which are mainly due to the substant…
Memory-Efficient Vision Transformers: An Activation-Aware Mixed-Rank Compression Strategy
Seyedarmin Azizi, Mahdi Nazemi, Massoud Pedram
As Vision Transformers (ViTs) increasingly set new benchmarks in computer vision, their practical deployment on inference engines is often hindered by their significant memory band…
Low-Precision Mixed-Computation Models for Inference on Edge
Seyedarmin Azizi, Mahdi Nazemi, Mehdi Kamal +1
This paper presents a mixed-computation neural network processing approach for edge applications that incorporates low-precision (low-width) Posit and low-precision fixed point (Fi…
Sensitivity-Aware Mixed-Precision Quantization and Width Optimization of Deep Neural Networks Through Cluster-Based Tree-Structured Parzen Estimation
Seyedarmin Azizi, Mahdi Nazemi, Arash Fayyazi +1
As the complexity and computational demands of deep learning models rise, the need for effective optimization methods for neural network designs becomes paramount. This work introd…
Training-Free Acceleration of ViTs with Delayed Spatial Merging
Jung Hwan Heo, Seyedarmin Azizi, Arash Fayyazi +1
Token merging has emerged as a new paradigm that can accelerate the inference of Vision Transformers (ViTs) without any retraining or fine-tuning. To push the frontier of training-…