6 papers · 1 filter
Robust Decentralized Fairness Auditing
Sayan Biswas, Jade Garcia Bourrée, Anne-Marie Kermarrec +2
Emerging legislation requires large language models (LLMs) to be audited for compliance with regulatory standards, particularly fairness. Such black-box audits typically assume a s…
The Evaluation Game: Beyond Static LLM Benchmarking
Paul Wang, Jade Garcia-Bourrée, Anne-Marie Kermarrec +1
As jailbreaks, adversarially crafted inputs that bypass safety constraints, continue to be discovered in Large Language Models, practitioners increasingly rely on fine-tuning as a…
Robust ML Auditing using Prior Knowledge
Jade Garcia Bourrée, Augustin Godinot, Martijn De Vos +5
Among the many technical challenges to enforcing AI regulations, one crucial yet underexplored problem is the risk of audit manipulation. This manipulation occurs when a platform d…
P2NIA: Privacy-Preserving Non-Iterative Auditing
Jade Garcia Bourrée, Hadrien Lautraite, Sébastien Gambs +3
The emergence of AI legislation has increased the need to assess the ethical compliance of high-risk AI systems. Traditional auditing methods rely on platforms' application program…
Fairness Auditing with Multi-Agent Collaboration
Martijn de Vos, Akash Dhasade, Jade Garcia Bourrée +4
Existing work in fairness auditing assumes that each audit is performed independently. In this paper, we consider multiple agents working together, each auditing the same platform…
Leveraging Imperfect Sources to Detect Fairwashing in Black-Box Auditing
Jade Garcia Bourrée, Erwan Le Merrer, Gilles Tredan +1
Algorithmic auditing has become central to platform accountability under frameworks such as the AI Act and the Digital Services Act. In practice, this obligation is discharged thro…