Gen-AI for User Safety: A Survey

Abstract

Machine Learning and data mining techniques (i.e. supervised and unsupervised techniques) are used across domains to detect user safety violations. Examples include classifiers used to detect whether an email is spam or a web-page is requesting bank login information. However, existing ML/DM classifiers are limited in their ability to understand natural languages w.r.t the context and nuances. The aforementioned challenges are overcome with the arrival of Gen-AI techniques, along with their inherent ability w.r.t translation between languages, fine-tuning between various tasks and domains. In this manuscript, we provide a comprehensive overview of the various work done while using Gen-AI techniques w.r.t user safety. In particular, we first provide the various domains (e.g. phishing, malware, content moderation, counterfeit, physical safety) across which Gen-AI techniques have been applied. Next, we provide how Gen-AI techniques can be used in conjunction with various data modalities i.e. text, images, videos, audio, executable binaries to detect violations of user-safety. Further, also provide an overview of how Gen-AI techniques can be used in an adversarial setting. We believe that this work represents the first summarization of Gen-AI techniques for user-safety.

References in corpus (17)

• Understanding the Characteristics of Internet Short Video Sharing: YouTube as a Case Study
• The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)
• A Survey of Malware Detection Using Deep Learning
• WaveFake: A Data Set to Facilitate Audio Deepfake Detection
• RatGPT: Turning online LLMs into Proxies for Malware Attacks
• Scaling Up LLM Reviews for Google Ads Content Moderation
• Pushing The Limit of LLM Capacity for Text Classification
• The World of Generative AI: Deepfakes and Large Language Models
• AI Safety in Generative AI Large Language Models: A Survey
• VisionGPT: LLM-Assisted Real-Time Anomaly Detection for Safe Visual Navigation
• EasyJailbreak: A Unified Framework for Jailbreaking Large Language Models
• MMIDR: Teaching Large Language Model to Interpret Multimodal Misinformation via Knowledge Distillation
• Multimodal Chain-of-Thought Reasoning via ChatGPT to Protect Children from Age-Inappropriate Apps
• Mission Impossible: A Statistical Perspective on Jailbreaking LLMs
• A Comprehensive Study on NLP Data Augmentation for Hate Speech Detection: Legacy Methods, BERT, and LLMs
• Towards Understanding Unsafe Video Generation
• DetoxBench: Benchmarking Large Language Models for Multitask Fraud & Abuse Detection