Distilling Safe LLM Systems via Soft Prompts for On Device Settings

Motasem Alfarra, Cristina Pinneri, Dana Kianfar +2

Deploying safe large language models (LLMs) on resource-constrained edge devices presents a critical challenge: while dual-model systems combining LLMs with guard models provide ef…