Gen-AI for User Safety: A Survey

Abstract

Machine Learning and data mining techniques (i.e. supervised and unsupervised techniques) are used across domains to detect user safety violations. Examples include classifiers used to detect whether an email is spam or a web-page is requesting bank login information. However, existing ML/DM classifiers are limited in their ability to understand natural languages w.r.t the context and nuances. The aforementioned challenges are overcome with the arrival of Gen-AI techniques, along with their inherent ability w.r.t translation between languages, fine-tuning between various tasks and domains. In this manuscript, we provide a comprehensive overview of the various work done while using Gen-AI techniques w.r.t user safety. In particular, we first provide the various domains (e.g. phishing, malware, content moderation, counterfeit, physical safety) across which Gen-AI techniques have been applied. Next, we provide how Gen-AI techniques can be used in conjunction with various data modalities i.e. text, images, videos, audio, executable binaries to detect violations of user-safety. Further, also provide an overview of how Gen-AI techniques can be used in an adversarial setting. We believe that this work represents the first summarization of Gen-AI techniques for user-safety.

References in corpus (29)

• Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
• Understanding the Characteristics of Internet Short Video Sharing: YouTube as a Case Study
• The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)
• Supervision Exists Everywhere: A Data Efficient Contrastive Language-Image Pre-training Paradigm
• Fake News in Sheep's Clothing: Robust Fake News Detection Against LLM-Empowered Style Attacks
• A Survey of Malware Detection Using Deep Learning
• AppPoet: Large Language Model based Android malware detection via multi-view prompt engineering
• LLM-Assisted Crisis Management: Building Advanced LLM Platforms for Effective Emergency Response and Public Collaboration
• WaveFake: A Data Set to Facilitate Audio Deepfake Detection
• Detecting Phishing Sites Using ChatGPT
• RatGPT: Turning online LLMs into Proxies for Malware Attacks
• Scaling Up LLM Reviews for Google Ads Content Moderation
• ChatSpamDetector: Leveraging Large Language Models for Effective Phishing Email Detection
• Pushing The Limit of LLM Capacity for Text Classification
• The World of Generative AI: Deepfakes and Large Language Models
• AI Safety in Generative AI Large Language Models: A Survey
• VITA: A Multi-modal LLM-based System for Longitudinal, Autonomous, and Adaptive Robotic Mental Well-being Coaching
• EasyJailbreak: A Unified Framework for Jailbreaking Large Language Models
• VisionGPT: LLM-Assisted Real-Time Anomaly Detection for Safe Visual Navigation
• MMIDR: Teaching Large Language Model to Interpret Multimodal Misinformation via Knowledge Distillation
• UnsafeBench: Benchmarking Image Safety Classifiers on Real-World and AI-Generated Images
• Is poisoning a real threat to LLM alignment? Maybe more so than you think
• LEMMA: Towards LVLM-Enhanced Multimodal Misinformation Detection with External Knowledge Augmentation
• A Comprehensive Study on NLP Data Augmentation for Hate Speech Detection: Legacy Methods, BERT, and LLMs
• Mission Impossible: A Statistical Perspective on Jailbreaking LLMs
• Moderating Illicit Online Image Promotion for Unsafe User-Generated Content Games Using Large Vision-Language Models
• Multimodal Chain-of-Thought Reasoning via ChatGPT to Protect Children from Age-Inappropriate Apps
• Towards Understanding Unsafe Video Generation
• DetoxBench: Benchmarking Large Language Models for Multitask Fraud & Abuse Detection