1 citations · 2 across the 9 of their papers we have counts for
5 papers · 1 filter
ClusterFewshot: Improving Few-shot Optimization for LLMs workflow
Omri Bar Haim, Shahar Katz, Lior Wolf
The performance of large language model (LLM) workflows often depends on selecting a small set of in-context demonstrations to guide model behavior on new tasks. Recent methods imp…
Detecting and Pruning Prominent but Detrimental Neurons in Large Language Models
Ameen Ali, Shahar Katz, Lior Wolf +1
Large language models (LLMs) often develop learned mechanisms specialized to specific datasets, such as reliance on domain-specific correlations, which yield high-confidence predic…
Segment-Based Attention Masking for GPTs
Shahar Katz, Liran Ringel, Yaniv Romano +1
Modern Language Models (LMs) owe much of their success to masked causal attention, the backbone of Generative Pre-Trained Transformer (GPT) models. Although GPTs can process the en…
Reversed Attention: On The Gradient Descent Of Attention Layers In GPT
Shahar Katz, Lior Wolf
The success of Transformer-based Language Models (LMs) stems from their attention mechanism. While this mechanism has been extensively studied in explainability research, particula…
Backward Lens: Projecting Language Model Gradients into the Vocabulary Space
Shahar Katz, Yonatan Belinkov, Mor Geva +1
Understanding how Transformer-based Language Models (LMs) learn and recall information is a key goal of the deep learning community. Recent interpretability methods project weights…