6 papers
TensorLens: End-to-End Transformer Analysis via High-Order Attention Tensors
Ido Andrew Atad, Itamar Zimerman, Shahar Katz +1
Attention matrices are fundamental to transformer research, supporting a broad range of applications including interpretability, visualization, manipulation, and distillation. Yet,…
AlignTree: Efficient Defense Against LLM Jailbreak Attacks
Gil Goren, Shahar Katz, Lior Wolf
Large Language Models (LLMs) are vulnerable to adversarial attacks that bypass safety guidelines and generate harmful content. Mitigating these vulnerabilities requires defense mec…
Execution Guided Line-by-Line Code Generation
Boaz Lavon, Shahar Katz, Lior Wolf
We present a novel approach to neural code generation that incorporates real-time execution signals into the language model generation process. While large language models (LLMs) h…
Detecting and Pruning Prominent but Detrimental Neurons in Large Language Models
Ameen Ali, Shahar Katz, Lior Wolf +1
Large language models (LLMs) often develop learned mechanisms specialized to specific datasets, such as reliance on domain-specific correlations, which yield high-confidence predic…
Segment-Based Attention Masking for GPTs
Shahar Katz, Liran Ringel, Yaniv Romano +1
Modern Language Models (LMs) owe much of their success to masked causal attention, the backbone of Generative Pre-Trained Transformer (GPT) models. Although GPTs can process the en…
Reversed Attention: On The Gradient Descent Of Attention Layers In GPT
Shahar Katz, Lior Wolf
The success of Transformer-based Language Models (LMs) stems from their attention mechanism. While this mechanism has been extensively studied in explainability research, particula…