410 citations · 964 across the 27 of their papers we have counts for
31 papers
CT: Counterfactual Causal Reasoning for Sentiment Shifts in Social-Media Conversation Trees
S M Rafiuddin, Atriya Sen
Sentiment in social-media threads does not only vary across posts; it shifts as users react to claims, corrections, evidence, and hostility within a branching reply tree. We study…
Gemma 4 Technical Report
Gemma Team, Sherif El Abd, Vaibhav Aggarwal +320
We introduce Gemma 4, a new generation of open-weight, natively multimodal language models in the Gemma model family. Designed to advance compute efficiency and reasoning, the Gemm…
The Bias is in the Details: An Assessment of Cognitive Bias in LLMs
R. Alexander Knipper, Charles S. Knipper, Kaiqi Zhang +3
As Large Language Models (LLMs) are increasingly embedded in real-world decision-making processes, it becomes crucial to examine the extent to which they exhibit cognitive biases.…
Seed-Free Synthetic Data Generation Framework for Instruction-Tuning LLMs: A Case Study in Thai
Parinthapat Pengpun, Can Udomcharoenchaikit, Weerayut Buaphet +1
We present a synthetic data approach for instruction-tuning large language models (LLMs) for low-resource languages in a data-efficient manner, specifically focusing on Thai. We id…
Gemma 2: Improving Open Language Models at a Practical Size
Gemma Team, Morgane Riviere, Shreya Pathak +195
In this work, we introduce Gemma 2, a new addition to the Gemma family of lightweight, state-of-the-art open models, ranging in scale from 2 billion to 27 billion parameters. In th…
BOND: Aligning LLMs with Best-of-N Distillation
Pier Giuseppe Sessa, Robert Dadashi, Léonard Hussenot +17
Reinforcement learning from human feedback (RLHF) is a key driver of quality and safety in state-of-the-art large language models. Yet, a surprisingly simple and strong inference-t…
Offline Regularised Reinforcement Learning for Large Language Models Alignment
Pierre Harvey Richemond, Yunhao Tang, Daniel Guo +15
The dominant framework for alignment of large language models (LLM), whether through reinforcement learning from human feedback or direct preference optimisation, is to learn from…
West-of-N: Synthetic Preferences for Self-Improving Reward Models
Alizée Pace, Jonathan Mallinson, Eric Malmi +2
The success of reinforcement learning from human feedback (RLHF) in language model alignment is strongly dependent on the quality of the underlying reward model. In this paper, we…
Small Language Models Improve Giants by Rewriting Their Outputs
Giorgos Vernikos, Arthur Bražinskas, Jakub Adamek +3
Despite the impressive performance of large language models (LLMs), they often lag behind specialized models in various tasks. LLMs only use a fraction of the existing training dat…
Teaching Small Language Models to Reason
Lucie Charlotte Magister, Jonathan Mallinson, Jakub Adamek +2
Chain of thought prompting successfully improves the reasoning capabilities of large language models, achieving state of the art results on a range of datasets. However, these reas…
Text Generation with Text-Editing Models
Eric Malmi, Yue Dong, Jonathan Mallinson +7
Text-editing models have recently become a prominent alternative to seq2seq models for monolingual text-generation tasks such as grammatical error correction, simplification, and s…
EdiT5: Semi-Autoregressive Text-Editing with T5 Warm-Start
Jonathan Mallinson, Jakub Adamek, Eric Malmi +1
We present EdiT5 - a novel semi-autoregressive text-editing model designed to combine the strengths of non-autoregressive text-editing and autoregressive decoding. EdiT5 is faster…
Controlled Text Generation as Continuous Optimization with Multiple Constraints
Sachin Kumar, Eric Malmi, Aliaksei Severyn +1
As large-scale language model pretraining pushes the state-of-the-art in text generation, recent work has turned to controlling attributes of the text such models generate. While m…
A Simple Recipe for Multilingual Grammatical Error Correction
Sascha Rothe, Jonathan Mallinson, Eric Malmi +2
This paper presents a simple recipe to train state-of-the-art multilingual Grammatical Error Correction (GEC) models. We achieve this by first proposing a language-agnostic method…
Unsupervised Text Style Transfer with Padded Masked Language Models
Eric Malmi, Aliaksei Severyn, Sascha Rothe
We propose Masker, an unsupervised text-editing method for style transfer. To tackle cases when no parallel source-target pairs are available, we train masked language models (MLMs…
Felix: Flexible Text Editing Through Tagging and Insertion
Jonathan Mallinson, Aliaksei Severyn, Eric Malmi +1
We present Felix --- a flexible text-editing approach for generation, designed to derive the maximum benefit from the ideas of decoding with bi-directional contexts and self-superv…
Encode, Tag, Realize: High-Precision Text Editing
Eric Malmi, Sebastian Krause, Sascha Rothe +2
We propose LaserTagger - a sequence tagging approach that casts text generation as a text editing task. Target texts are reconstructed from the inputs using three main edit operati…
Leveraging Pre-trained Checkpoints for Sequence Generation Tasks
Sascha Rothe, Shashi Narayan, Aliaksei Severyn
Unsupervised pre-training of large neural models has recently revolutionized Natural Language Processing. By warm-starting from the publicly released checkpoints, NLP practitioners…
Breaking the Softmax Bottleneck via Learnable Monotonic Pointwise Non-linearities
Octavian-Eugen Ganea, Sylvain Gelly, Gary Bécigneul +1
The Softmax function on top of a final linear layer is the de facto method to output probability distributions in neural networks. In many applications such as language models or t…
Adversarial Neural Networks for Cross-lingual Sequence Tagging
Heike Adel, Anton Bryl, David Weiss +1
We study cross-lingual sequence tagging with little or no labeled data in the target language. Adversarial training has previously been shown to be effective for training cross-lin…
On Accurate Evaluation of GANs for Language Generation
Stanislau Semeniuta, Aliaksei Severyn, Sylvain Gelly
Generative Adversarial Networks (GANs) are a promising approach to language generation. The latest works introducing novel GAN models for language generation use n-gram based metri…
Prosody Modifications for Question-Answering in Voice-Only Settings
Aleksandr Chuklin, Aliaksei Severyn, Johanne Trippas +3
Many popular form factors of digital assistants---such as Amazon Echo, Apple Homepod, or Google Home---enable the user to hold a conversation with these systems based only on the s…
Eval all, trust a few, do wrong to none: Comparing sentence generation models
Ondřej Cífka, Aliaksei Severyn, Enrique Alfonseca +1
In this paper, we study recent neural generative models for text generation related to variational autoencoders. Previous works have employed various techniques to control the prio…
Avoiding Your Teacher's Mistakes: Training Neural Networks with Controlled Weak Supervision
Mostafa Dehghani, Aliaksei Severyn, Sascha Rothe +1
Training deep neural networks requires massive amounts of training data, but for many tasks only limited labeled data is available. This makes weak supervision attractive, using we…