54 citations · 55 across the 10 of their papers we have counts for
12 papers
The Missing Minimal Pair: Stereotype Evaluation in LLMs
Nataliya Stepanova, Ivan Titov, Emily Allaway +1
A common approach to measuring bias in Large Language Models is to compare the log-likelihoods of two contrastive stereotype sentences. We argue that such single-pair comparisons a…
Analyzing LLM Instruction Optimization for Tabular Fact Verification
Xiaotang Du, Giwon Hong, Wai-Chung Kwan +4
Instruction optimization provides a lightweight, model-agnostic approach to enhancing the reasoning performance of large language models (LLMs). This paper presents the first syste…
Operationalising the Superficial Alignment Hypothesis via Task Complexity
Tomás Vergara-Browne, Darshan Patil, Ivan Titov +3
The superficial alignment hypothesis (SAH) posits that large language models learn most of their knowledge during pre-training, and that post-training merely surfaces this knowledg…
Anthropomimetic Uncertainty: What Verbalized Uncertainty in Language Models is Missing
Dennis Ulmer, Alexandra Lorson, Ivan Titov +1
Human users increasingly communicate with large language models (LLMs), but LLMs suffer from frequent overconfidence in their output, even when its accuracy is questionable, which…
What's New in My Data? Novelty Exploration via Contrastive Generation
Masaru Isonuma, Ivan Titov
Fine-tuning is widely used to adapt language models for specific goals, often leveraging real-world data such as patient records, customer-service interactions, or web content in l…
Post-hoc Disentanglement of Textual and Acoustic Features in Pre-trained Speech Encoders
Hosein Mohebbi, Grzegorz Chrupała, Willem Zuidema +2
Pre-trained speech models build entangled internal representations, which capture a variety of features (e.g., pitch, loudness, syntax, or semantics of an utterance) in a distribut…
Solovay reducibility implies S2a-reducibility
Ivan Titov
The original notion of Solovay reducibility was introduced by Robert M. Solovay (unpublished notes) in 1975 as a measure of relative randomness. The S2a-reducibility introduced by…
Extending the Limit Theorem of Barmpalias and Lewis-Pye to all reals
Ivan Titov
By a celebrated result of Kučera and Slaman (DOI:10.1137/S0097539799357441), the Martin-Löf random left-c.e. reals form the highest left-c.e. Solovay degree. Barmpalias and Lewis-P…
Unlearning Traces the Influential Training Data of Language Models
Masaru Isonuma, Ivan Titov
Identifying the training datasets that influence a language model's outputs is essential for minimizing the generation of harmful content and enhancing its performance. Ideally, we…
Subspace Chronicles: How Linguistic Information Emerges, Shifts and Interacts during Language Model Training
Max Müller-Eberstein, Rob van der Goot, Barbara Plank +1
Representational spaces learned via language modeling are fundamental to Natural Language Processing (NLP), however there has been limited understanding regarding how and when duri…
On the Transferability of Visually Grounded PCFGs
Yanpeng Zhao, Ivan Titov
There has been a significant surge of interest in visually grounded grammar induction in recent times. While a variety of models have been developed for the task and have demonstra…
Emergence of Language with Multi-agent Games: Learning to Communicate with Sequences of Symbols
Serhii Havrylov, Ivan Titov
Learning to communicate through interaction, rather than relying on explicit supervision, is often considered a prerequisite for developing a general AI. We study a setting where t…