18 papers · 1 filter
Stress-testing Alignment Midtraining
Sid Baines, Jonathan Bostock, Maria Angelica Martinez +3
When aligning frontier models through post-training techniques, it is not possible to directly demonstrate all of the behaviours we want a model to exhibit in all possible deployme…
Inoculation Midtraining with Learned Neologisms
Kyle O'Brien, Edward James Young, Puria Radmard +4
Large language models (LLMs) often learn both desirable and undesirable properties during post-training. We study whether midtraining, an earlier training stage, can shape which of…
Last Translation Benchmark
Vilém Zouhar, Niyati Bafna, Mukund Choudhary +263
For scientific progress, we need benchmarks that test the limits of state-of-the-art models, and evaluation methods that inform us about failure cases. As models get stronger, stan…
When Role-playing, Do Models Believe What They Say?
Benjamin Sturgeon, David Africa, Sid Black
Language models can state that "the Earth orbits the Sun" and, when role-playing Aristotle, assert the opposite. Recent work argues that persona adoption is fundamental to how lang…
PACUTE: Phonology-, Affix-, and Character-level Understanding of Tokens for Filipino
Jann Railey Montalan, David Demitri Africa, Jimson Paulo Layacan +3
Large language models (LLMs) process text as sequences of subword tokens, which can obscure the character-level and morphological structure that underlies word formation. This limi…
Consistency Training Can Entrench Misalignment
David Demitri Africa, Arathi Mani
Consistency training encourages a model to produce similar outputs across related inputs or sampling procedures. Such methods are simple, scalable, and largely label-free, but thei…
Consistency Training while Mitigating Obfuscation via Rate Matching
Sohaib Imran, Prakhar Gupta, Jannes Elstner +1
Large language models are often influenced by extraneous input features, such as cues revealing a user's preferred answer. Consistency training reduces this influence by training m…
LURE: Live-Usage Replay Evaluations for Reducing Evaluation Awareness
Igor Ivanov, David Demitri Africa
Large language models can recognize when they are being evaluated (evaluation awareness) and behave differently because of that, which undermines the validity of safety and alignme…
Alignment Pretraining: AI Discourse Causes Self-Fulfilling (Mis)alignment
Cameron Tice, Puria Radmard, Samuel Ratnam +3
Pretraining corpora contain extensive discourse about AI systems, yet the causal influence of this discourse on downstream alignment remains poorly understood. If prevailing descri…
Inoculation Prompting: Eliciting traits from LLMs during training can suppress them at test-time
Daniel Tan, Anders Woodruff, Niels Warncke +4
Language model finetuning often results in learning undesirable traits in combination with desired ones. To address this, we propose inoculation prompting: modifying finetuning dat…
Steering Awareness: Detecting Activation Steering from Within
Joshua Fonseca Rivera, David Demitri Africa
Activation steering -- adding a vector to a model's residual stream to modify its behavior -- is widely used in safety evaluations as if the model cannot detect the intervention. W…
Meta-Pretraining for Zero-Shot Cross-Lingual Named Entity Recognition in Low-Resource Philippine Languages
David Demitri Africa, Suchir Salhan, Yuval Weiss +2
Named-entity recognition (NER) in low-resource languages is usually tackled by finetuning very large multilingual LMs, an option that is often infeasible in memory- or latency-cons…
Investigating ReLoRA: Effects on the Learning Dynamics of Small Language Models
Yuval Weiss, David Demitri Africa, Paula Buttery +1
Parameter-efficient methods like LoRA have revolutionised large language model (LLM) fine-tuning. ReLoRA extends this idea to pretraining by repeatedly merging and reinitialising l…
Pico: A Modular Framework for Hypothesis-Driven Small Language Model Research
Richard Diehl Martinez, David Demitri Africa, Yuval Weiss +3
Building language models (LMs), especially small and medium ones, remains more art than science. While large LMs often improve by sheer scale, it is still unclear why many design c…
No Answer Needed: Predicting LLM Answer Accuracy from Question-Only Linear Probes
Iván Vicente Moreno Cencerrado, Arnau Padrés Masdemont, Anton Gonzalvez Hawthorne +2
Do large language models (LLMs) anticipate when they will answer correctly? To study this, we extract activations after a question is read but before any tokens are generated, and…
Learning Dynamics of Meta-Learning in Small Model Pretraining
David Demitri Africa, Yuval Weiss, Paula Buttery +1
Large language models are powerful but costly. We ask whether meta-learning can make the pretraining of small language models not only better but also more interpretable. We integr…
Identifying a Circuit for Verb Conjugation in GPT-2
David Demitri Africa
I implement a procedure to isolate and interpret the sub-network (or "circuit") responsible for subject-verb agreement in GPT-2 Small. In this study, the model is given prompts whe…
Batayan: A Filipino NLP benchmark for evaluating Large Language Models
Jann Railey Montalan, Jimson Paulo Layacan, David Demitri Africa +5
Recent advances in large language models (LLMs) have demonstrated remarkable capabilities on widely benchmarked high-resource languages. However, linguistic nuances of under-resour…