From the 1 of 4 linked papers with an AI index.
4 papers
A dataset of rated conceptual arguments
Emery Cooper, Caspar Oesterheld, Linh Chi Nguyen +2
The paper introduces a dataset of 951 expert‑rated argumentative critiques on 442 position texts covering AI safety, decision theory, ethics, and politics, and uses it to benchmark…
ConvApparel: A Benchmark Dataset and Validation Framework for User Simulators in Conversational Recommenders
Ofer Meshi, Krisztian Balog, Sally Goldman +5
The promise of LLM-based user simulators to improve conversational AI is hindered by a critical "realism gap," leading to systems that are optimized for simulated interactions, but…
Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety
Tomek Korbak, Mikita Balesni, Elizabeth Barnes +38
AI systems that "think" in human language offer a unique opportunity for AI safety: we can monitor their chains of thought (CoT) for the intent to misbehave. Like all other known A…
A dataset of questions on decision-theoretic reasoning in Newcomb-like problems
Caspar Oesterheld, Emery Cooper, Miles Kodama +2
We introduce a dataset of natural-language questions in the decision theory of so-called Newcomb-like problems. Newcomb-like problems include, for instance, decision problems in wh…