58 citations · 186 across the 30 of their papers we have counts for
6 papers · 1 filter
How Context Shapes Truth: Geometric Transformations of Statement-level Truth Representations in LLMs
Shivam Adarsh, Maria Maistro, Christina Lioma
Large Language Models (LLMs) often encode whether a statement is true as a vector in their residual stream activations. These vectors, also known as truth vectors, have been studie…
Correcting Gradient-Based Circuit Localization via Interaction-Aware Backpropagation
Joakim Edin, Casper L. Christensen, Róbert Csordás +5
Circuit localization methods aim to identify the subset of model components responsible for specific behaviors in large language models, enabling detailed mechanistic analysis. Mos…
As easy as PIE: understanding when pruning causes language models to disagree
Pietro Tropeano, Maria Maistro, Tuukka Ruotsalo +1
Language Model (LM) pruning compresses the model by removing weights, nodes, or other parts of its architecture. Typically, pruning focuses on the resulting efficiency gains at the…
A Reality Check on Context Utilisation for Retrieval-Augmented Generation
Lovisa Hagström, Sara Vera Marjanović, Haeun Yu +5
Retrieval-augmented generation (RAG) helps address the limitations of parametric knowledge embedded within a language model (LM). In real world settings, retrieved information can…
DYNAMICQA: Tracing Internal Knowledge Conflicts in Language Models
Sara Vera Marjanović, Haeun Yu, Pepa Atanasova +3
Knowledge-intensive language understanding tasks require Language Models (LMs) to integrate relevant context, mitigating their inherent weaknesses, such as incomplete or outdated k…
Multi-Head Self-Attention with Role-Guided Masks
Dongsheng Wang, Casper Hansen, Lucas Chaves Lima +4
The state of the art in learning meaningful semantic representations of words is the Transformer model and its attention mechanisms. Simply put, the attention mechanisms learn to a…