Auditing Information Disclosure During Large-Scale Gradient-Based Training via Gradient Uniqueness

Sleem Abdelghafar, Maryam Aliakbarpour, Chris Jermaine +1

Auditing information disclosure across every datapoint during the training of LLMs is challenging. We propose a principled, attack-agnostic approach that uses mutual information to…