Privacy-Aware Document Visual Question Answering

Abstract

Document Visual Question Answering (DocVQA) has quickly grown into a central task of document understanding. But despite the fact that documents contain sensitive or copyrighted information, none of the current DocVQA methods offers strong privacy guarantees. In this work, we explore privacy in the domain of DocVQA for the first time, highlighting privacy issues in state of the art multi-modal LLM models used for DocVQA, and explore possible solutions. Specifically, we focus on invoice processing as a realistic document understanding scenario, and propose a large scale DocVQA dataset comprising invoice documents and associated questions and answers. We employ a federated learning scheme, that reflects the real-life distribution of documents in different businesses, and we explore the use case where the data of the invoice provider is the sensitive information to be protected. We demonstrate that non-private models tend to memorise, a behaviour that can lead to exposing private information. We then evaluate baseline training schemes employing federated learning and differential privacy in this multi-modal scenario, where the sensitive information might be exposed through either or both of the two input modalities: vision (document image) or language (OCR tokens). Finally, we design attacks exploiting the memorisation effect of the model, and demonstrate their effectiveness in probing a representative DocVQA models.

35 pages, 12 figures, accepted for publication at the 18th International Conference on Document Analysis and Recognition, ICDAR 2024

References in corpus (20)

• Deep Learning with Differential Privacy
• LoRA: Low-Rank Adaptation of Large Language Models
• Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning
• iDLG: Improved Deep Leakage from Gradients
• Quantifying Memorization Across Neural Language Models
• Large Language Models Can Be Strong Differentially Private Learners
• Label Leakage and Protection in Two-party Split Learning
• Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Models
• Unlocking High-Accuracy Differentially Private Image Classification through Scale
• Toward Training at ImageNet Scale with Differential Privacy
• Preventing Verbatim Memorization in Language Models Gives a False Sense of Privacy
• Subject Membership Inference Attacks in Federated Learning
• Group privacy for personalized federated learning
• Computing Tight Differential Privacy Guarantees Using FFT
• Document Visual Question Answering Challenge 2020
• Subject Granular Differential Privacy in Federated Learning
• Fine-Tuning with Differential Privacy Necessitates an Additional Hyperparameter Search
• On the Efficacy of Differentially Private Few-shot Image Classification
• M^4I: Multi-modal Models Membership Inference
• Practical Membership Inference Attacks Against Large-Scale Multi-Modal Models: A Pilot Study