31 citations · 82 across the 31 of their papers we have counts for
34 papers
AI Persuasion as a Threat to Human Control
Joshua Levy, Mick Yang, Kellin Pelrine
The threat that AI persuasion poses to human control has been acknowledged in the literature, but not yet systematically studied. Now that persuasion attacks are no longer theoreti…
AI Security Leaderboard: Methodology, Results and Minimal Standard
Jasper Timm, Lukas Struppek, Ziwei Xu +12
The AI Security Leaderboard is an independent benchmark that ranks the safeguards of frontier AI models from least to most secure. It tests models against the FARAI Minimal Stan…
Concept Influence: Leveraging Interpretability to Improve Performance and Efficiency in Training Data Attribution
Matthew Kowal, Goncalo Paulo, Louis Jaburi +6
As large language models are increasingly trained and fine-tuned, practitioners need methods to identify which training data drive specific behaviors, particularly unintended ones.…
Exposing the Systematic Vulnerability of Open-Weight Models to Prefill Attacks
Lukas Struppek, Adam Gleave, Kellin Pelrine
As the capabilities of large language models continue to advance, so does their potential for misuse. While closed-source models typically rely on external defenses, open-weight mo…
TamperBench: Systematically Stress-Testing LLM Safety Under Fine-Tuning and Tampering
Saad Hossain, Tom Tseng, Punya Syon Pandey +8
As increasingly capable open-weight large language models (LLMs) are deployed, improving their tamper resistance against unsafe modifications, whether accidental or intentional, be…
Large language models can effectively convince people to believe conspiracies
Thomas H. Costello, Kellin Pelrine, Matthew Kowal +6
Large language models (LLMs) have been shown to be persuasive across a variety of contexts. But it remains unclear whether this persuasive power advantages accuracy, or if bad acto…
Emergent Persuasion: Will LLMs Persuade Without Being Prompted?
Vincent Chang, Thee Ho, Sunishchal Dev +4
With the wide-scale adoption of conversational AI systems, AI are now able to exert unprecedented influence on human opinion and beliefs. Recent work has shown that many Large Lang…
: A Social Media User Dataset for LLM Persona Evaluation and Training
Aurélien Bück-Kaeffer, Je Qin Chooi, Dan Zhao +5
Large language models (LLMs) offer promising capabilities for simulating social media dynamics at scale, enabling studies that would be ethically or logistically challenging with h…
CrediBench: Building Web-Scale Network Datasets for Information Integrity
Emma Kondrup, Sebastian Sabry, Hussein Abdallah +9
Automatically assessing the credibility of online sources presents an invaluable tool for navigating today's information ecosystem. However, existing approaches either depend on sc…
Jailbreak-Tuning: Models Efficiently Learn Jailbreak Susceptibility
Brendan Murphy, Dillon Bowen, Shahrad Mohammadzadeh +4
AI systems are rapidly advancing in capability, and frontier model developers broadly acknowledge the need for safeguards against serious misuse. However, this paper demonstrates t…
Veracity: An Open-Source AI Fact-Checking System
Taylor Lynn Curtis, Maximilian Puelma Touzel, William Garneau +8
The proliferation of misinformation poses a significant threat to society, exacerbated by the capabilities of generative AI. This demo paper introduces Veracity, an open-source AI…
It's the Thought that Counts: Evaluating the Attempts of Frontier LLMs to Persuade on Harmful Topics
Matthew Kowal, Jasper Timm, Jean-Francois Godbout +6
Persuasion is a powerful capability of large language models (LLMs) that both enables beneficial applications (e.g. helping people quit smoking) and raises significant risks (e.g.…
The Structural Safety Generalization Problem
Julius Broomfield, Tom Gibbs, Ethan Kosak-Hine +7
LLM jailbreaks are a widespread safety challenge. Given this problem has not yet been tractable, we suggest targeting a key failure mechanism: the failure of safety to generalize a…
Accidental Vulnerability: Factors in Fine-Tuning that Shift Model Safeguards
Punya Syon Pandey, Samuel Simko, Kellin Pelrine +1
As large language models (LLMs) gain popularity, their vulnerability to adversarial attacks emerges as a primary concern. While fine-tuning models on domain-specific datasets is of…
From Intuition to Understanding: Using AI Peers to Overcome Physics Misconceptions
Ruben Weijers, Denton Wu, Hannah Betts +10
Generative AI has the potential to transform personalization and accessibility of education. However, it raises serious concerns about accuracy and helping students become independ…
Epistemic Integrity in Large Language Models
Bijean Ghafouri, Shahrad Mohammadzadeh, James Zhou +8
Large language models are increasingly relied upon as sources of information, but their propensity for generating false or misleading statements with high confidence poses risks fo…
A Guide to Misinformation Detection Data and Evaluation
Camille Thibault, Jacob-Junqi Tian, Gabrielle Peloquin-Skulski +7
Misinformation is a complex societal issue, and mitigating solutions are difficult to create due to data deficiencies. To address this, we have curated the largest collection of (m…
A Simulation System Towards Solving Societal-Scale Manipulation
Maximilian Puelma Touzel, Sneheel Sarangi, Austin Welch +12
The rise of AI-driven manipulation poses significant risks to societal trust and democratic processes. Yet, studying these effects in real-world settings at scale is ethically and…
Web Retrieval Agents for Evidence-Based Misinformation Detection
Jacob-Junqi Tian, Hao Yu, Yury Orlovskiy +7
This paper develops an agent-based automated fact-checking approach for detecting misinformation. We demonstrate that combining a powerful LLM agent, which does not have access to…
Emerging Vulnerabilities in Frontier Models: Multi-Turn Jailbreak Attacks
Tom Gibbs, Ethan Kosak-Hine, George Ingebretsen +6
Large language models (LLMs) are improving at an exceptional rate. However, these models are still susceptible to jailbreak attacks, which are becoming increasingly dangerous as mo…
Scaling Trends for Data Poisoning in LLMs
Dillon Bowen, Brendan Murphy, Will Cai +3
LLMs produce harmful and undesirable behavior when trained on datasets containing even a small fraction of poisoned data. We demonstrate that GPT models remain vulnerable to fine-t…
Regional and Temporal Patterns of Partisan Polarization during the COVID-19 Pandemic in the United States and Canada
Zachary Yang, Anne Imouza, Maximilian Puelma Touzel +6
Public health measures were among the most polarizing topics debated online during the COVID-19 pandemic. Much of the discussion surrounded specific events, such as when and which…
Can Go AIs be adversarially robust?
Tom Tseng, Euan McLean, Kellin Pelrine +2
Prior work found that superhuman Go AIs can be defeated by simple adversarial strategies, especially "cyclic" attacks. In this paper, we study whether adding natural countermeasure…
Combining Confidence Elicitation and Sample-based Methods for Uncertainty Quantification in Misinformation Mitigation
Mauricio Rivera, Jean-François Godbout, Reihaneh Rabbany +1
Large Language Models have emerged as prime candidates to tackle misinformation mitigation. However, existing approaches struggle with hallucinations and overconfident predictions.…