PROPARAG: An Evidence-Grounded Decision Support System for Cybersecurity Policy Assessment

Abstract

Cybersecurity policy assessment is an evidence-intensive organizational decision task. Reviewers must locate relevant policy statements, determine whether they sufficiently address security controls, identify missing elements, and decide where further policy action is required. Existing AI-based approaches support parts of this process, but often provide limited support for evidence verification, partial coverage, and diagnostic review. We present PROPARAG, an evidence-grounded decision-support framework for control-level cybersecurity policy assessment. PROPARAG retrieves relevant policy evidence, assigns full, partial, or absent coverage, identifies policy gaps, generates corrective recommendations, and provides evidence-linked explanations for expert review. We evaluate the framework on 1,007 NIST SP~800-53 controls across two real-world organizational policy corpora. PROPARAG achieves F1-scores of 88.54% and 82.31%, and outperforms the strongest evaluated baseline. Semantic evidence retrieval provides substantial gains, while structured assessment further improves performance over single-stage analysis. Expert evaluation also shows strong gap correctness and evidence groundedness. The results suggest that effective policy-assessment support also benefits from evidence provenance, explicit intermediate states, diagnostic information, and human reviewability. Evidence provenance, explicit intermediate states, structured diagnosis, and human reviewability are important parts of the assessment process.