The Boy Who Survived: Removing Harry Potter from an LLM is harder than reported

Adam Shostack

Recent work arXiv.2310.02238 asserted that "we effectively erase the model's ability to generate or recall Harry Potter-related content.'' This claim is shown to be overbroad. A sm…