5 papers
LANTERN: Illuminating Hidden Mathematical Knowledge in Language Models
Pavel Tikhonov, Elena Tutubalina, Ivan Oseledets +2
Language models can now prove theorems, but people still decide which problems to pursue. We ask whether a model's internal representations can help identify promising mathematical…
Your Transformer Can Hold Two Thoughts at Once: Evidence of Linear Superposition in LLMs
Pavel Tikhonov, Anton Korznikov, Matvey Mikhalchuk +6
While Large Language Models (LLMs) rely on highly non-linear components, in this work we demonstrate that they exhibit fundamental linearity: when inputs from distinct text streams…
One Task Vector is not Enough: A Large-Scale Study for In-Context Learning
Pavel Tikhonov, Ivan Oseledets, Elena Tutubalina
In-context learning (ICL) enables Large Language Models (LLMs) to adapt to new tasks using few examples, with task vectors - specific hidden state activations - hypothesized to enc…
SumHiS: Extractive Summarization Exploiting Hidden Structure
Tikhonov Pavel, Anastasiya Ianina, Valentin Malykh
Extractive summarization is a task of highlighting the most important parts of the text. We introduce a new approach to extractive summarization task using hidden clustering struct…
WikiMulti: a Corpus for Cross-Lingual Summarization
Pavel Tikhonov, Valentin Malykh
Cross-lingual summarization (CLS) is the task to produce a summary in one particular language for a source document in a different language. We introduce WikiMulti - a new dataset…