4 citations · 28 across the 86 of their papers we have counts for
9 papers · 2 filters
From Stories to Features: Scaling Concept Vectors for AI Alignment & Interpretability
Dmitry Scherbakov, Oleg Chernikov, Victoria Khamash +5
We study feature vectors: directions in a language model's activation space that correspond to human-interpretable concepts, such as honesty or deception. Existing unsupervised met…
AI Consilium: a Benchmark of LLM Clinical Multidisciplinary Second Opinions
Alexey A. Bokov, Ivan I. Panin, Nikolay B. Nechaev +10
Large language models (LLMs) can generate differential diagnoses for complex cases, but a useful second opinion must also be factually grounded, clinically coherent, actionable, an…
E-MoE: Enhanced Mixture-of-Experts for Non-Factorized Diffusion Language Models
Arseny Ivanov, Alexander Kolesov, Alexander Korotin +2
Masked diffusion models (MDMs) generate sequences by progressively unmasking several tokens per denoising step, but their reverse process is typically factorized over positions, li…
LANTERN: Illuminating Hidden Mathematical Knowledge in Language Models
Pavel Tikhonov, Elena Tutubalina, Ivan Oseledets +2
Language models can now prove theorems, but people still decide which problems to pursue. We ask whether a model's internal representations can help identify promising mathematical…
Your Transformer Can Hold Two Thoughts at Once: Evidence of Linear Superposition in LLMs
Pavel Tikhonov, Anton Korznikov, Matvey Mikhalchuk +6
While Large Language Models (LLMs) rely on highly non-linear components, in this work we demonstrate that they exhibit fundamental linearity: when inputs from distinct text streams…
Evaluating Losslessness in Speculative Decoding Under Finite-Precision Inference
Ilya Koziev, Leonid Sinev, Ivan Oseledets
Lossless speculative decoding is typically defined at the algorithmic level: a speculative procedure proposes multiple tokens and a verification procedure is designed to preserve t…
Structured Transforms for Low-Overhead Quantization of Language Models
Daria Cherniuk, Alexander Rudikov, Boris Kashin +1
We revisit Kashin-decomposition-based weight quantization for large language models and propose an improved algorithm with stronger convergence properties and structured, efficient…
LLM-Guided Evolution for Medical Decision Pipelines
Ivan Sviridov, Artem Oskin, Ivan Panin +4
Adapting large language models (LLMs) to clinical workflows often requires costly fine-tuning or manual prompt and pipeline engineering. We study LLM-guided MAP-Elites evolution as…
OCC-RAG: Optimal Cognitive Core for Faithful Question Answering
Maksim Savkin, Mikhail Goncharov, Alexander Gambashidze +7
Recent progress in the development of language models has been defined by scale, with each generation absorbing more of the world's knowledge into its weights. However, many practi…