Example-Based Concept Analysis Framework for Deep Weather Forecast Models

Abstract

To improve the trustworthiness of an AI model, finding consistent, understandable representations of its inference process is essential. This understanding is particularly important in high-stakes operations such as weather forecasting, where the identification of underlying meteorological mechanisms is as critical as the accuracy of the predictions. Despite the growing literature that addresses this issue through explainable AI, the applicability of their solutions is often limited due to their AI-centric development. To fill this gap, we follow a user-centric process to develop an example-based concept analysis framework, which identifies cases that follow a similar inference process as the target instance in a target model and presents them in a user-comprehensible format. Our framework provides the users with visually and conceptually analogous examples, including the probability of concept assignment to resolve ambiguities in weather mechanisms. To bridge the gap between vector representations identified from models and human-understandable explanations, we compile a human-annotated concept dataset and implement a user interface to assist domain experts involved in the the framework development.

39 pages, 10 figures

References in corpus (18)

• Skillful Precipitation Nowcasting using Deep Generative Models of Radar
• Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
• The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
• Captum: A unified and generic model interpretability library for PyTorch
• Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)
• Physically Interpretable Neural Networks for the Geosciences: Applications to Earth System Variability
• Dynamic Channel Pruning: Feature Boosting and Suppression
• Understanding intermediate layers using linear classifier probes
• Towards Automatic Concept-based Explanations
• Effective Training Strategies for Deep-learning-based Precipitation Nowcasting and Estimation
• The Internal State of an LLM Knows When It's Lying
• Concept Embedding Analysis: A Review
• Bridging the Human-AI Knowledge Gap: Concept Discovery and Transfer in AlphaZero
• Explainable AI-Based Interface System for Weather Forecasting Model
• Gaussian Process Probes (GPP) for Uncertainty-Aware Probing
• MTSMAE: Masked Autoencoders for Multivariate Time-Series Forecasting
• Understanding Distributed Representations of Concepts in Deep Neural Networks without Supervision
• Probing Network Decisions: Capturing Uncertainties and Unveiling Vulnerabilities Without Label Information