2.7k citations · 2.9k across the 18 of their papers we have counts for
21 papers
What do Reward Models Memorize?
Ivo Verhoeven, Pushkar Mishra, Ekaterina Shutova
This paper studies what discriminatively trained reward models (RMs) memorize by measuring counterfactual memorization on two human preference datasets. We show that RMs 1) misallo…
Yesterday's News: Benchmarking Multi-Dimensional Out-of-Distribution Generalization of Misinformation Detection Models
Ivo Verhoeven, Pushkar Mishra, Ekaterina Shutova
This article introduces misinfo-general, a benchmark dataset for evaluating misinformation models' ability to perform out-of-distribution generalization. Misinformation changes rap…
A (More) Realistic Evaluation Setup for Generalisation of Community Models on Malicious Content Detection
Ivo Verhoeven, Pushkar Mishra, Rahel Beloch +2
Community models for malicious content detection, which take into account the context from a social graph alongside the content itself, have shown remarkable performance on benchma…
Llama 2: Open Foundation and Fine-Tuned Chat Models
Hugo Touvron, Louis Martin, Kevin Stone +65
In this work, we develop and release Llama 2, a collection of pretrained and fine-tuned large language models (LLMs) ranging in scale from 7 billion to 70 billion parameters. Our f…
Investigating the Robustness of Sequential Recommender Systems Against Training Data Perturbations
Filippo Betello, Federico Siciliano, Pushkar Mishra +1
Sequential Recommender Systems (SRSs) are widely employed to model user behavior over time. However, their robustness in the face of perturbations in training data remains a largel…
Scientific and Creative Analogies in Pretrained Language Models
Tamara Czinczoll, Helen Yannakoudakis, Pushkar Mishra +1
This paper examines the encoding of analogy in large-scale pretrained language models, such as BERT and GPT-2. Existing analogy datasets typically focus on a limited set of analogi…