5 papers · 1 filter
The Sharp Directions Are Against You: Curvature Analysis of Activation Steering
Seyedarmin Azizi, Arya Fayyazi, Parsa Razmara +3
Activation steering adds a vector to a language model's internal representations at inference time. It is a lightweight alternative to fine-tuning for behavioral control. The stand…
Running Fewer Loops Removes Refusal Behaviour in Recurrent-Depth Language Models
Seyedarmin Azizi, Erfan Baghaei Potraghloo, Arya Fayyazi +2
A recurrent-depth language model reuses one block of weights several times in a row. The number of repetitions, which we call loops, is chosen when the model runs, not when it is t…
Labels Override Definitions in Jev-Style Typed Decision Models
Seyedarmin Azizi, Erfan Baghaei Potraghloo, Massoud Pedram
A typed decision model answers a fixed question about an input by returning a probability for each of several caller-defined options. Each option carries a short label and a writte…
SkipKV: Selective Skipping of KV Generation and Storage for Efficient Inference with Large Reasoning Models
Jiayi Tian, Seyedarmin Azizi, Yequan Zhao +7
Large reasoning models (LRMs) often incur significant key-value (KV) cache overhead, due to their linear growth with the verbose chain-of-thought (CoT) reasoning. This incurs both…
Activation Steering for Chain-of-Thought Compression
Seyedarmin Azizi, Erfan Baghaei Potraghloo, Massoud Pedram
Large language models (LLMs) excel at complex reasoning when they include intermediate steps, known as "chains of thought" (CoTs). However, these rationales are often overly verbos…