How Much Do Code Language Models Remember? An Investigation on Data Extraction Attacks before and after Fine-tuning

Fabio Salerno, Ali Al-Kaswan, Maliheh Izadi

Code language models, while widely popular, are often trained on unsanitized source code gathered from across the Internet. Previous work revealed that pre-trained models can remem…