On the Opportunities and Risks of Foundation Models

Abstract

AI is undergoing a paradigm shift with the rise of models (e.g., BERT, DALL-E, GPT-3) that are trained on broad data at scale and are adaptable to a wide range of downstream tasks. We call these models foundation models to underscore their critically central yet incomplete character. This report provides a thorough account of the opportunities and risks of foundation models, ranging from their capabilities (e.g., language, vision, robotics, reasoning, human interaction) and technical principles(e.g., model architectures, training procedures, data, systems, security, evaluation, theory) to their applications (e.g., law, healthcare, education) and societal impact (e.g., inequity, misuse, economic and environmental impact, legal and ethical considerations). Though foundation models are based on standard deep learning and transfer learning, their scale results in new emergent capabilities,and their effectiveness across so many tasks incentivizes homogenization. Homogenization provides powerful leverage but demands caution, as the defects of the foundation model are inherited by all the adapted models downstream. Despite the impending widespread deployment of foundation models, we currently lack a clear understanding of how they work, when they fail, and what they are even capable of due to their emergent properties. To tackle these questions, we believe much of the critical research on foundation models will require deep interdisciplinary collaboration commensurate with their fundamentally sociotechnical nature.

arXiv commentAuthored by the Center for Research on Foundation Models (CRFM) at the Stanford Institute for Human-Centered Artificial Intelligence (HAI). Report page with citation guidelines: https://crfm.stanford.edu/report.html

References in corpus (255)

• Efficient Estimation of Word Representations in Vector Space
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Auto-Encoding Variational Bayes
• Distilling the Knowledge in a Neural Network
• TensorFlow: A system for large-scale machine learning
• Overcoming catastrophic forgetting in neural networks
• BioBERT: a pre-trained biomedical language representation model for biomedical text mining
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Denoising Diffusion Probabilistic Models
• Learning Transferable Visual Models From Natural Language Supervision
• Prototypical Networks for Few-shot Learning
• WaveNet: A Generative Model for Raw Audio
• Large Scale GAN Training for High Fidelity Natural Image Synthesis
• Bootstrap your own latent: A new approach to self-supervised Learning
• Towards A Rigorous Science of Interpretable Machine Learning
• Language Models are Few-Shot Learners
• Axiomatic Attribution for Deep Networks
• Neural Discrete Representation Learning
• LoRA: Low-Rank Adaptation of Large Language Models
• Learning Important Features Through Propagating Activation Differences
• Longformer: The Long-Document Transformer
• Shortcut Learning in Deep Neural Networks
• Improved Baselines with Momentum Contrastive Learning
• Learning to Plan Chemical Syntheses
• Predicting Cardiovascular Risk Factors from Retinal Fundus Photographs using Deep Learning
• Microsoft COCO Captions: Data Collection and Evaluation Server
• Scaling Laws for Neural Language Models
• Learning agile and dynamic motor skills for legged robots
• Evaluating Large Language Models Trained on Code
• MLP-Mixer: An all-MLP Architecture for Vision
• Frustratingly Easy Domain Adaptation
• Deep Unsupervised Learning using Nonequilibrium Thermodynamics
• Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings
• NICE: Non-linear Independent Components Estimation
• Understanding Black-box Predictions via Influence Functions
• EfficientNetV2: Smaller Models and Faster Training
• Zero-Shot Text-to-Image Generation
• Generative Modeling by Estimating Gradients of the Data Distribution
• Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning
• The Diversity-Innovation Paradox in Science
• Mixed Precision Training
• Linformer: Self-Attention with Linear Complexity
• Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
• Junction Tree Variational Autoencoder for Molecular Graph Generation
• CTRL: A Conditional Transformer Language Model for Controllable Generation
• Snorkel: Rapid Training Data Creation with Weak Supervision
• Learning Differentially Private Recurrent Language Models
• Poisoning Attacks against Support Vector Machines
• Dynamic Memory Networks for Visual and Textual Question Answering
• What Makes for Good Views for Contrastive Learning?
• Solving Rubik's Cube with a Robot Hand
• On the Cross-lingual Transferability of Monolingual Representations
• Multilingual Denoising Pre-training for Neural Machine Translation
• Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimization
• Conservative Q-Learning for Offline Reinforcement Learning
• REALM: Retrieval-Augmented Language Model Pre-Training
• Training Deep Nets with Sublinear Memory Cost
• Adversarially Robust Generalization Requires More Data
• BAE: BERT-based Adversarial Examples for Text Classification
• The Pile: An 800GB Dataset of Diverse Text for Language Modeling
• Generating Long Sequences with Sparse Transformers
• Semi-supervised Sequence Learning
• A General Approach for Predicting the Behavior of the Supreme Court of the United States
• The Measure and Mismeasure of Fairness
• Decision Transformer: Reinforcement Learning via Sequence Modeling
• Using Pre-Training Can Improve Model Robustness and Uncertainty
• Visualizing Deep Neural Network Decisions: Prediction Difference Analysis
• Generalisation in humans and deep neural networks
• Deep Learning Scaling is Predictable, Empirically
• Massively Multitask Networks for Drug Discovery
• Do ImageNet Classifiers Generalize to ImageNet?
• Emergent Tool Use From Multi-Agent Autocurricula
• Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity
• Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization
• Mockingjay: Unsupervised Speech Representation Learning with Deep Bidirectional Transformer Encoders
• GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding
• VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Text
• Learning Latent Dynamics for Planning from Pixels
• Learning How to Ask: Querying LMs with Mixtures of Soft Prompts
• Reformer: The Efficient Transformer
• Safe Model-based Reinforcement Learning with Stability Guarantees
• AI2-THOR: An Interactive 3D Environment for Visual AI
• Visual Reinforcement Learning with Imagined Goals
• Invariant Risk Minimization
• XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization
• Towards a Critical Race Methodology in Algorithmic Fairness
• Prefix-Tuning: Optimizing Continuous Prompts for Generation
• Release Strategies and the Social Impacts of Language Models
• What do Neural Machine Translation Models Learn about Morphology?
• Contrastive Learning of Medical Visual Representations from Paired Images and Text
• Variational Information Maximisation for Intrinsically Motivated Reinforcement Learning
• One Model To Learn Them All
• Model compression via distillation and quantization
• MOPO: Model-based Offline Policy Optimization
• Cognitive Science in the era of Artificial Intelligence: A roadmap for reverse-engineering the infant language-learner
• Fairness Without Demographics in Repeated Loss Minimization
• Perceiver IO: A General Architecture for Structured Inputs & Outputs
• The Evolved Transformer
• True Few-Shot Learning with Language Models
• Unifying Vision-and-Language Tasks via Text Generation
• Sparse Networks from Scratch: Faster Training without Losing Performance
• Stochastic Neural Networks for Hierarchical Reinforcement Learning
• Natural Adversarial Examples
• Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets
• Unsupervised Domain Adaptation through Self-Supervision
• GraphCodeBERT: Pre-training Code Representations with Data Flow
• Predictive Inequity in Object Detection
• Learning and Evaluating General Linguistic Intelligence
• Data Shapley: Equitable Valuation of Data for Machine Learning
• Tracking the World State with Recurrent Entity Networks
• Scaling Laws for Autoregressive Generative Modeling
• Time-Aware Language Models as Temporal Knowledge Bases
• Privacy Preserving Distributed Machine Learning with Federated Learning
• Algorithmic Monoculture and Social Welfare
• Quantifying the Carbon Emissions of Machine Learning
• Carbon Emissions and Large Neural Network Training
• ThreeDWorld: A Platform for Interactive Multi-Modal Physical Simulation
• Artificial Intelligence in Clinical Health Care Applications: Viewpoint
• Performance Impact Caused by Hidden Bias of Training Data for Recognizing Textual Entailment
• Parameter Efficient Training of Deep Convolutional Neural Networks by Dynamic Sparse Reparameterization
• Multilingual is not enough: BERT for Finnish
• Scalable agent alignment via reward modeling: a research direction
• Rethinking Attention with Performers
• Understanding intermediate layers using linear classifier probes
• Reinforcement Learning in Healthcare: A Survey
• Carbontracker: Tracking and Predicting the Carbon Footprint of Training Deep Learning Models
• The Dataset Nutrition Label: A Framework To Drive Higher Data Quality Standards
• Pretrained Transformers as Universal Computation Engines
• A Kernel Theory of Modern Data Augmentation
• HYPE: A Benchmark for Human eYe Perceptual Evaluation of Generative Models
• CUAD: An Expert-Annotated NLP Dataset for Legal Contract Review
• Thieves on Sesame Street! Model Extraction of BERT-based APIs
• On the Effect of Dropping Layers of Pre-trained Transformer Models
• Unsolved Problems in ML Safety
• PatentBERT: Patent Classification with Fine-Tuning a pre-trained BERT Model
• Multimodal Few-Shot Learning with Frozen Language Models
• Theoretical Analysis of Self-Training with Deep Networks on Unlabeled Data
• Charformer: Fast Character Transformers via Gradient-based Subword Tokenization
• An Investigation of Why Overparameterization Exacerbates Spurious Correlations
• Overlearning Reveals Sensitive Attributes
• Query2box: Reasoning over Knowledge Graphs in Vector Space using Box Embeddings
• Towards Vision-Based Smart Hospitals: A System for Tracking and Monitoring Hand Hygiene Compliance
• Keeping Community in the Loop: Understanding Wikipedia Stakeholder Values for Machine Learning-Based Systems
• Calibrate Before Use: Improving Few-Shot Performance of Language Models
• Structured Pruning of a BERT-based Question Answering Model
• Making AI Forget You: Data Deletion in Machine Learning
• Causal Mediation Analysis for Interpreting Neural NLP: The Case of Gender Bias
• On the Theory of Transfer Learning: The Importance of Task Diversity
• From Language to Goals: Inverse Reinforcement Learning for Vision-Based Instruction Following
• You Autocomplete Me: Poisoning Vulnerabilities in Neural Code Completion
• Language-Agnostic Representation Learning of Source Code from Structure and Context
• Learning to Understand Goal Specifications by Modelling Reward
• Data and its (dis)contents: A survey of dataset development and use in machine learning research
• Robot Learning in Homes: Improving Generalization and Reducing Dataset Bias
• Noise or Signal: The Role of Image Backgrounds in Object Recognition
• ZeRO-Infinity: Breaking the GPU Memory Wall for Extreme Scale Deep Learning
• GamePad: A Learning Environment for Theorem Proving
• Kernel and Rich Regimes in Overparametrized Models
• Few-Shot Learning via Learning the Representation, Provably
• MERLOT: Multimodal Neural Script Knowledge Models
• Variational Inverse Control with Events: A General Framework for Data-Driven Reward Definition
• Understanding the Failure Modes of Out-of-Distribution Generalization
• Guaranteeing Safety for Neural Network-Based Aircraft Collision Avoidance Systems
• QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question Answering
• Graph-based, Self-Supervised Program Repair from Diagnostic Feedback
• Predicting What You Already Know Helps: Provable Self-Supervised Learning
• From Optimizing Engagement to Measuring Value
• The Role of Cooperation in Responsible AI Development
• Energy Usage Reports: Environmental awareness as part of algorithmic accountability
• Characterising Bias in Compressed Models
• Model Patching: Closing the Subgroup Performance Gap with Data Augmentation
• Context-Aware Legal Citation Recommendation using Deep Learning
• Generative Language Modeling for Automated Theorem Proving
• Bootleg: Chasing the Tail with Self-Supervised Named Entity Disambiguation
• Modifying Memories in Transformer Models
• Towards Ecologically Valid Research on Language User Interfaces
• Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Loss
• Alignment of Language Agents
• Learning perturbation sets for robust machine learning
• On Adversarial Bias and the Robustness of Fair Machine Learning
• Learning to summarize from human feedback
• BEHAVIOR: Benchmark for Everyday Household Activities in Virtual, Interactive, and Ecological Environments
• Addressing "Documentation Debt" in Machine Learning Research: A Retrospective Datasheet for BookCorpus
• Explaining Landscape Connectivity of Low-cost Solutions for Multilayer Nets
• MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scale
• Multiple Interactions Made Easy (MIME): Large Scale Demonstrations Data for Imitation
• Understanding Self-supervised Learning with Dual Deep Networks
• Why Do Pretrained Language Models Help in Downstream Tasks? An Analysis of Head and Prompt Tuning
• Pretraining Representations for Data-Efficient Reinforcement Learning
• BERT Goes to Law School: Quantifying the Competitive Advantage of Access to Large Legal Corpora in Contract Understanding
• HTLM: Hyper-Text Pre-Training and Prompting of Language Models
• End-to-End Robotic Reinforcement Learning without Reward Engineering
• Accuracy on the Line: On the Strong Correlation Between Out-of-Distribution and In-Distribution Generalization
• Scalable Second Order Optimization for Deep Learning
• Combiner: Full Attention Transformer with Sparse Computation Cost
• Ludwig: a type-based declarative deep learning toolbox
• Break-It-Fix-It: Unsupervised Learning for Program Repair
• Contrastive Learning Inverts the Data Generating Process
• Facts as Experts: Adaptable and Interpretable Neural Memory over Symbolic Knowledge
• A Benchmark for Lease Contract Review
• Simulating Action Dynamics with Neural Process Networks
• A Neural Topic-Attention Model for Medical Term Abbreviation Disambiguation
• How much is Wikipedia Lagging Behind News?
• Risks from Learned Optimization in Advanced Machine Learning Systems
• Overton: A Data System for Monitoring and Improving Machine-Learned Products
• Poisoning the Unlabeled Dataset of Semi-Supervised Learning
• Viewmaker Networks: Learning Views for Unsupervised Representation Learning
• Adversarial Reprogramming of Neural Networks
• Top-KAST: Top-K Always Sparse Training
• Cloudy with high chance of DBMS: A 10-year prediction for Enterprise-Grade ML
• Revealing Persona Biases in Dialogue Systems
• Poisoning and Backdooring Contrastive Learning
• BREEDS: Benchmarks for Subpopulation Shift
• The iWildCam 2021 Competition Dataset
• CraftAssist: A Framework for Dialogue-enabled Interactive Agents
• Implicit regularization for deep neural networks driven by an Ornstein-Uhlenbeck like process
• Contrastive estimation reveals topic posterior information to linear models
• Training Agents using Upside-Down Reinforcement Learning
• Shape Matters: Understanding the Implicit Bias of the Noise Covariance
• Model Assertions for Monitoring and Improving ML Models
• C5T5: Controllable Generation of Organic Molecules with Transformers
• Affirmative Algorithms: The Legal Grounds for Fairness as Awareness
• Anticipating Safety Issues in E2E Conversational AI: Framework and Tooling
• Fast, Structured Clinical Documentation via Contextual Autocomplete
• Dynaboard: An Evaluation-As-A-Service Platform for Holistic Next-Generation Benchmarking
• OPAL: Offline Primitive Discovery for Accelerating Offline Reinforcement Learning
• HULK: An Energy Efficiency Benchmark Platform for Responsible Natural Language Processing
• MathBERT: A Pre-trained Language Model for General NLP Tasks in Mathematics Education
• When Does Pretraining Help? Assessing Self-Supervised Learning for Law and the CaseHOLD Dataset
• ESR: Ethics and Society Review of Artificial Intelligence Research
• Domain-Specific Pretraining for Vertical Search: Case Study on Biomedical Literature
• Persuasive Natural Language Generation -- A Literature Review
• Interpretable Multi-Step Reasoning with Knowledge Extraction on Complex Healthcare Question Answering
• Mandoline: Model Evaluation under Distribution Shift
• Fair Machine Learning Under Partial Compliance
• The Benchmark Lottery
• A Theory of Label Propagation for Subpopulation Shift
• Editable Neural Networks
• ProtoTransformer: A Meta-Learning Approach to Providing Student Feedback
• Label Noise SGD Provably Prefers Flat Global Minimizers
• Benchmarking Differential Privacy and Federated Learning for BERT Models
• On the proper role of linguistically-oriented deep net analysis in linguistic theorizing
• Proof Artifact Co-training for Theorem Proving with Language Models
• DABS: A Domain-Agnostic Benchmark for Self-Supervised Learning
• Causal Analysis of Agent Behavior for AI Safety
• Using Transformers to Provide Teachers with Personalized Feedback on their Classroom Discourse: The TalkMoves Application
• Impossibility results for fair representations
• Performance in the Courtroom: Automated Processing and Visualization of Appeal Court Decisions in France
• Training a First-Order Theorem Prover from Synthetic Data
• Distributed Deep Learning in Open Collaborations
• When to reply? Context Sensitive Models to Predict Instructor Interventions in MOOC Forums
• Cross-neutralising: Probing for joint encoding of linguistic information in multilingual models
• The Law of Large Documents: Understanding the Structure of Legal Contracts Using Visual Cues
• Feedback Effects in Repeat-Use Criminal Risk Assessments
• How could Neural Networks understand Programs?

Cited by in corpus (198)

• Learning to Prompt for Vision-Language Models
• Generative AI
• DINOv2: Learning Robust Visual Features without Supervision
• SpectralGPT: Spectral Remote Sensing Foundation Model
• Multitask Prompted Training Enables Zero-Shot Task Generalization
• The Ethics of ChatGPT in Medicine and Healthcare: A Systematic Review on Large Language Models (LLMs)
• Opportunities and Challenges for ChatGPT and Large Language Models in Biomedicine and Health
• Self-Supervised Speech Representation Learning: A Review
• Large language models in medicine: the potentials and pitfalls
• Sustainable AI: Environmental Implications, Challenges and Opportunities
• Structured Pruning for Deep Convolutional Neural Networks: A survey
• Florence: A New Foundation Model for Computer Vision
• Human heuristics for AI-generated language are flawed
• TSMixer: Lightweight MLP-Mixer Model for Multivariate Time Series Forecasting
• Large Legal Fictions: Profiling Legal Hallucinations in Large Language Models
• The Programmer's Assistant: Conversational Interaction with a Large Language Model for Software Development
• The Metacognitive Demands and Opportunities of Generative AI
• Co-Writing with Opinionated Language Models Affects Users' Views
• Large AI Models in Health Informatics: Applications, Challenges, and the Future
• CodeAid: Evaluating a Classroom Deployment of an LLM-based Programming Assistant that Balances Student and Educator Needs
• Compute and Energy Consumption Trends in Deep Learning Inference
• Vision-Language Models for Medical Report Generation and Visual Question Answering: A Review
• A Comprehensive Survey of Deep Transfer Learning for Anomaly Detection in Industrial Time Series: Methods, Applications, and Directions
• A Taxonomy of Prompt Modifiers for Text-To-Image Generation
• Geo-knowledge-guided GPT models improve the extraction of location descriptions from disaster-related social media messages
• Multimodal Data Integration for Oncology in the Era of Deep Neural Networks: A Review
• Auditing of AI: Legal, Ethical and Technical Approaches
• A Survey on Unsupervised Anomaly Detection Algorithms for Industrial Images
• Taking the Next Step with Generative Artificial Intelligence: The Transformative Role of Multimodal Large Language Models in Science Education
• CenterCLIP: Token Clustering for Efficient Text-Video Retrieval
• Harms from Increasingly Agentic Algorithmic Systems
• Machine Culture
• FAIR for AI: An interdisciplinary and international community building perspective
• From Image to Language: A Critical Analysis of Visual Question Answering (VQA) Approaches, Challenges, and Opportunities
• A Study on the Implementation of Generative AI Services Using an Enterprise Data-Based LLM Application Architecture
• Unsolved Problems in ML Safety
• General Purpose Artificial Intelligence Systems (GPAIS): Properties, Definition, Taxonomy, Societal Implications and Responsible Governance
• Can an Embodied Agent Find Your "Cat-shaped Mug"? LLM-Guided Exploration for Zero-Shot Object Navigation
• Addressing Bias in Generative AI: Challenges and Research Opportunities in Information Management
• Large Language Models Can Be Strong Differentially Private Learners
• Physics-informed machine learning for building performance simulation-A review of a nascent field
• Finetuned Language Models Are Zero-Shot Learners
• Learning from models beyond fine-tuning
• Merak: An Efficient Distributed DNN Training Framework with Automated 3D Parallelism for Giant Foundation Models
• AstroCLIP: A Cross-Modal Foundation Model for Galaxies
• Domain Adversarial Spatial-Temporal Network: A Transferable Framework for Short-term Traffic Forecasting across Cities
• Exploring the Role of AI Assistants in Computer Science Education: Methods, Implications, and Instructor Perspectives
• Defining the scope of AI regulations
• Anatomy of an AI-powered malicious social botnet
• MERLOT: Multimodal Neural Script Knowledge Models
• Zeno: An Interactive Framework for Behavioral Evaluation of Machine Learning
• Reinforcement Learning for Generative AI: State of the Art, Opportunities and Open Research Challenges
• Hyperspectral unmixing for Raman spectroscopy via physics-constrained autoencoders
• ERNIE-GeoL: A Geography-and-Language Pre-trained Model and its Applications in Baidu Maps
• Fine-tuning and Utilization Methods of Domain-specific LLMs
• The Impact of ChatGPT and LLMs on Medical Imaging Stakeholders: Perspectives and Use Cases
• Debiasing Methods for Fairer Neural Models in Vision and Language Research: A Survey
• FoodSAM: Any Food Segmentation
• Exploring Adapter-based Transfer Learning for Recommender Systems: Empirical Studies and Practical Insights
• A Security Risk Taxonomy for Prompt-Based Interaction With Large Language Models
• Multimodal Foundation Models for Material Property Prediction and Discovery
• The worst of both worlds: A comparative analysis of errors in learning from data in psychology and machine learning
• From Screens to Scenes: A Survey of Embodied AI in Healthcare
• Transfer Language Selection for Zero-Shot Cross-Lingual Abusive Language Detection
• Pre-trained Language Models in Biomedical Domain: A Systematic Survey
• The Survey on Multi-Source Data Fusion in Cyber-Physical-Social Systems:Foundational Infrastructure for Industrial Metaverses and Industries 5.0
• Normative Challenges of Risk Regulation of Artificial Intelligence and Automated Decision-Making
• Self-supervised representations in speech-based depression detection
• Zero-shot cross-lingual transfer language selection using linguistic similarity
• BadPre: Task-agnostic Backdoor Attacks to Pre-trained NLP Foundation Models
• On the design space between molecular mechanics and machine learning force fields
• The Clever Hans Effect in Unsupervised Learning
• Message Ritual: A Posthuman Account of Living with Lamp
• Clinical Insights: A Comprehensive Review of Language Models in Medicine
• Dual Use Concerns of Generative AI and Large Language Models
• CLOOB: Modern Hopfield Networks with InfoLOOB Outperform CLIP
• Comparing Sentence-Level Suggestions to Message-Level Suggestions in AI-Mediated Communication
• Domain-specific ChatBots for Science using Embeddings
• Federated Few-Shot Learning for Mobile NLP
• Masked Particle Modeling on Sets: Towards Self-Supervised High Energy Physics Foundation Models
• FeatureNeRF: Learning Generalizable NeRFs by Distilling Foundation Models
• On the Sample Complexity of Quantum Boltzmann Machine Learning
• PEFT-SER: On the Use of Parameter Efficient Transfer Learning Approaches For Speech Emotion Recognition Using Pre-trained Speech Models
• Opportunities for Large Language Models and Discourse in Engineering Design
• Building Privacy-Preserving and Secure Geospatial Artificial Intelligence Foundation Models
• Position is Power: System Prompts as a Mechanism of Bias in Large Language Models (LLMs)
• Probing Speech Emotion Recognition Transformers for Linguistic Knowledge
• RadFusion: Benchmarking Performance and Fairness for Multimodal Pulmonary Embolism Detection from CT and EHR
• Leveraging large language models for structured information extraction from pathology reports
• Embedded Visual Prompt Tuning
• Toward parallel intelligence: an interdisciplinary solution for complex systems
• Probing transfer learning with a model of synthetic correlated datasets
• A Holistic Assessment of the Carbon Footprint of Noor, a Very Large Arabic Language Model
• Will Code Remain a Relevant User Interface for End-User Programming with Generative AI Models?
• The Dawn of AI-Native EDA: Opportunities and Challenges of Large Circuit Models
• Perceptron Theory Can Predict the Accuracy of Neural Networks
• Welcome Your New AI Teammate: On Safety Analysis by Leashing Large Language Models
• M6-10T: A Sharing-Delinking Paradigm for Efficient Multi-Trillion Parameter Pretraining
• ReCycle: Resilient Training of Large DNNs using Pipeline Adaptation
• Mapping the Landscape of Generative AI in Network Monitoring and Management
• Foundation Models and Transformers for Anomaly Detection: A Survey
• Jack and Masters of all Trades: One-Pass Learning Sets of Model Sets From Large Pre-Trained Models
• Mapping the individual, social, and biospheric impacts of Foundation Models
• Perceived Trustworthiness of Natural Language Generators
• SeisT: A foundational deep learning model for earthquake monitoring tasks
• Engineering Artificial Intelligence: Framework, Challenges, and Future Direction
• Low-resource finetuning of foundation models beats state-of-the-art in histopathology
• Findings of the The RuATD Shared Task 2022 on Artificial Text Detection in Russian
• LLM-Mediated Domain-Specific Voice Agents: The Case of TextileBot
• How Will It Drape Like? Capturing Fabric Mechanics from Depth Images
• Tucano: Advancing Neural Text Generation for Portuguese
• Rethinking model prototyping through the MedMNIST+ dataset collection
• What makes a language easy to deep-learn? Deep neural networks and humans similarly benefit from compositional structure
• Differentiable Physics: A Position Piece
• Ranking and Tuning Pre-trained Models: A New Paradigm for Exploiting Model Hubs
• Foundational Models and Federated Learning: Survey, Taxonomy, Challenges and Practical Insights
• INTERN: A New Learning Paradigm Towards General Vision
• Perturbing Attention Gives You More Bang for the Buck: Subtle Imaging Perturbations That Efficiently Fool Customized Diffusion Models
• Undesirable Biases in NLP: Addressing Challenges of Measurement
• A Survey of Behavior Learning Applications in Robotics -- State of the Art and Perspectives
• Towards Foundation Models for Materials Science: The Open MatSci ML Toolkit
• Towards Symbolic XAI -- Explanation Through Human Understandable Logical Relationships Between Features
• FAIR AI Models in High Energy Physics
• Emergency Department Decision Support using Clinical Pseudo-notes
• Towards Foundation Model for Chemical Reactor Modeling: Meta-Learning with Physics-Informed Adaptation
• Intent Tagging: Exploring Micro-Prompting Interactions for Supporting Granular Human-GenAI Co-Creation Workflows
• Parameter-Efficient Learning for Text-to-Speech Accent Adaptation
• Data Poisoning Vulnerabilities Across Healthcare AI Architectures: A Security Threat Analysis
• Practically implementing an LLM-supported collaborative vulnerability remediation process: a team-based approach
• CoLLIE: Continual Learning of Language Grounding from Language-Image Embeddings
• ForgetMe: Evaluating Selective Forgetting in Generative Models
• Adapting an ASR Foundation Model for Spoken Language Assessment
• Distribution-based Emotion Recognition in Conversation
• Analytical Engines With Context-Rich Processing: Towards Efficient Next-Generation Analytics
• Evaluating the Fairness of Discriminative Foundation Models in Computer Vision
• Integrating Emotion Recognition with Speech Recognition and Speaker Diarisation for Conversations
• Shaking the foundations: delusions in sequence models for interaction and control
• Pixelated Butterfly: Simple and Efficient Sparse training for Neural Network Models
• How to Estimate Model Transferability of Pre-Trained Speech Models?
• Preemptively Pruning Clever-Hans Strategies in Deep Neural Networks
• LLM-Assisted Visual Analytics: Opportunities and Challenges
• Should attention be all we need? The epistemic and ethical implications of unification in machine learning
• Adaptive Intelligence: leveraging insights from adaptive behavior in animals to build flexible AI systems
• Relational Programming with Foundation Models
• TADA: Task-Agnostic Dialect Adapters for English
• A Survey of Spatio-Temporal EEG data Analysis: from Models to Applications
• Tissue Concepts: supervised foundation models in computational pathology
• Generalist Models in Medical Image Segmentation: A Survey and Performance Comparison with Task-Specific Approaches
• Generalized Decision Transformer for Offline Hindsight Information Matching
• SMILE: Evaluation and Domain Adaptation for Social Media Language Understanding
• Solving Probability and Statistics Problems by Program Synthesis
• Stuck-at Faults in ReRAM Neuromorphic Circuit Array and their Correction through Machine Learning
• MaskTerial: A Foundation Model for Automated 2D Material Flake Detection
• Saturn Platform: Foundation Model Operations and Generative AI for Financial Services
• A Scalable Model Specialization Framework for Training and Inference using Submodels and its Application to Speech Model Personalization
• Towards Collaborative Plan Acquisition through Theory of Mind Modeling in Situated Dialogue
• ESG Accountability Made Easy: DocQA at Your Service
• 3D Object Detection and High-Resolution Traffic Parameters Extraction Using Low-Resolution LiDAR Data
• On Parallelism in Music and Language: A Perspective from Symbol Emergence Systems based on Probabilistic Generative Models
• AgentArcEval: An Architecture Evaluation Method for Foundation Model based Agents
• Insulin Resistance Prediction From Wearables and Routine Blood Biomarkers
• Optimizing Context-Enhanced Relational Joins
• On the attribution of confidence to large language models
• Fine-tuning Strategies for Domain Specific Question Answering under Low Annotation Budget Constraints
• Language Models as a Knowledge Source for Cognitive Agents
• ArtPerception: ASCII Art-based Jailbreak on LLMs with Recognition Pre-test
• A Good Foundation is Worth Many Labels: Label-Efficient Panoptic Segmentation
• Design considerations for a hierarchical semantic compositional framework for medical natural language understanding
• Bridging Text and Crystal Structures: Literature-driven Contrastive Learning for Materials Science
• Risks of AI Foundation Models in Education
• How to Compliment a Human -- Designing Affective and Well-being Promoting Conversational Things
• Utilizing Grounded SAM for self-supervised frugal camouflaged human detection
• Scaling Law for Recommendation Models: Towards General-purpose User Representations
• Towards More Robust NLP System Evaluation: Handling Missing Scores in Benchmarks
• Reconstructing hadronically decaying tau leptons with a jet foundation model
• Beyond Partisan Leaning: A Comparative Analysis of Political Bias in Large Language Models
• Toward Foundation Models for Earth Monitoring: Proposal for a Climate Change Benchmark
• Cross-Lingual Language Model Meta-Pretraining
• A Systematic Review of Open Datasets Used in Text-to-Image (T2I) Gen AI Model Safety
• PØDA: Prompt-driven Zero-shot Domain Adaptation
• One to Transfer All: A Universal Transfer Framework for Vision Foundation Model with Few Data
• 10 Security and Privacy Problems in Large Foundation Models
• Measure Twice, Cut Once: Quantifying Bias and Fairness in Deep Neural Networks
• Artificial Intelligence in Mechanical Engineering: Foundations, Opportunities, Challenges, and Responsibilities
• CrossedWires: A Dataset of Syntactically Equivalent but Semantically Disparate Deep Learning Models
• Efficient Rotation Invariance in Deep Neural Networks through Artificial Mental Rotation
• A Novel Information-Theoretic Objective to Disentangle Representations for Fair Classification
• Six misconceptions about large language models: A minimal model and diagnostic taxonomy
• Deep Learning for Accurate Vision-based Catch Composition in Tropical Tuna Purse Seiners
• Radio Galaxy Zoo: Morphological classification by Fanaroff-Riley designation using self-supervised pre-training
• Collaborating Foundation Models for Domain Generalized Semantic Segmentation
• The TYC Dataset for Understanding Instance-Level Semantics and Motions of Cells in Microstructures
• Active Learning at the ImageNet Scale
• Accelerating Deep Learning with Dynamic Data Pruning
• ÚFAL at MultiLexNorm 2021: Improving Multilingual Lexical Normalization by Fine-tuning ByT5
• Who Decides if AI is Fair? The Labels Problem in Algorithmic Auditing
• An argument for the impossibility of machine intelligence
• Bag-of-Vectors Autoencoders for Unsupervised Conditional Text Generation