4 papers · 1 filter
Logical Consistency Between Disagreeing Experts and Its Role in AI Safety
Andrés Corrada-Emmanuel
If two experts disagree on a test, we may conclude both cannot be 100 per cent correct. But if they completely agree, no possible evaluation can be excluded. This asymmetry in the…
No-Knowledge Alarms for Misaligned LLMs-as-Judges
Andrés Corrada-Emmanuel
If we use LLMs as judges to evaluate the complex decisions of other LLMs, who or what monitors the judges? Infinite monitoring chains are inevitable whenever we do not know the gro…
Algebraic Evaluation Theorems
Andrés Corrada-Emmanuel
Majority voting (MV) is the prototypical ``wisdom of the crowd'' algorithm. Theorems considering when MV is optimal for group decisions date back to Condorcet's 1785 jury \emph{dec…
The logic of NTQR evaluations of noisy AI agents: Complete postulates and logically consistent error correlations
Andrés Corrada-Emmanuel
In his "ship of state" allegory (\textit{Republic}, Book VI, 488) Plato poses a question -- how can a crew of sailors presumed to know little about the art of navigation recognize…