Explainability Through Systematicity: The Hard Systematicity Challenge for Artificial Intelligence

Abstract

This paper argues that explainability is only one facet of a broader ideal that shapes our expectations towards artificial intelligence (AI). Fundamentally, the issue is to what extent AI exhibits systematicity--not merely in being sensitive to how thoughts are composed of recombinable constituents, but in striving towards an integrated body of thought that is consistent, coherent, comprehensive, and parsimoniously principled. This richer conception of systematicity has been obscured by the long shadow of the "systematicity challenge" to connectionism, according to which network architectures are fundamentally at odds with what Fodor and colleagues termed "the systematicity of thought." I offer a conceptual framework for thinking about "the systematicity of thought" that distinguishes four senses of the phrase. I use these distinctions to defuse the perceived tension between systematicity and connectionism and show that the conception of systematicity that historically shaped our sense of what makes thought rational, authoritative, and scientific is more demanding than the Fodorian notion. To determine whether we have reason to hold AI models to this ideal of systematicity, I then argue, we must look to the rationales for systematization and explore to what extent they transfer to AI models. I identify five such rationales and apply them to AI. This brings into view the "hard systematicity challenge." However, the demand for systematization itself needs to be regulated by the rationales for systematization. This yields a dynamic understanding of the need to systematize thought, which tells us how systematic we need AI models to be and when.

39 pages; final, published version

References in corpus (19)

• Self-Consistency Improves Chain of Thought Reasoning in Language Models
• Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)
• Meaning without reference in large language models
• ML Interpretability: Simple Isn't Easy
• Still No Lie Detector for Language Models: Probing Empirical and Conceptual Roadblocks
• Measuring and Narrowing the Compositionality Gap in Language Models
• Internal Consistency and Self-Feedback in Large Language Models: A Survey
• Training Large Language Models to Reason in a Continuous Latent Space
• A Philosophical Introduction to Language Models - Part II: The Way Forward
• Assessing Composition in Sentence Vector Representations
• Universal Self-Consistency for Large Language Model Generation
• A Comprehensive Guide to Explainable AI: From Classical Models to LLMs
• Can AI Rely on the Systematicity of Truth? The Challenge of Modelling Normative Domains
• Gerrymandering Individual Fairness
• On Explainability in AI-Solutions: A Cross-Domain Survey
• PLANNER: Generating Diversified Paragraph via Latent Language Diffusion Model
• Propositional Interpretability in Artificial Intelligence
• Enhancing Logical Reasoning in Large Language Models through Graph-based Synthetic Data
• Episodic Memories Generation and Evaluation Benchmark for Large Language Models