Training Convolutional Neural Networks with the Forward-Forward algorithm

Abstract

Recent successes in image analysis with deep neural networks are achieved almost exclusively with Convolutional Neural Networks (CNNs), typically trained using the backpropagation (BP) algorithm. In a 2022 preprint, Geoffrey Hinton proposed the Forward-Forward (FF) algorithm as a biologically inspired alternative, where positive and negative examples are jointly presented to the network and training is guided by a locally defined goodness function. Here, we extend the FF paradigm to CNNs. We introduce two spatially extended labeling strategies, based on Fourier patterns and morphological transformations, that enable convolutional layers to access label information across all spatial positions. On CIFAR10, we show that deeper FF-trained CNNs can be optimized successfully and that morphology-based labels prevent shortcut solutions on dataset with more complex and fine features. On CIFAR100, carefully designed label sets scale effectively to 100 classes. Class Activation Maps reveal that FF-trained CNNs learn meaningful and complementary features across layers. Together, these results demonstrate that FF training is feasible beyond fully connected networks, provide new insights into its learning dynamics and stability, and highlight its potential for neuromorphic computing and biologically inspired learning.

PEER-REVIEWED VERSION PUBLISHED ON "SCIENTIFIC REPORTS" (2025) DOI: 10.1038/s41598-025-26235-2

References in corpus (21)

• A Hebbian/Anti-Hebbian Neural Network for Linear Subspace Learning: A Derivation from Multidimensional Scaling of Streaming Data
• Fine-Tuning Language Models with Just Forward Passes
• Gradients without Backpropagation
• Hebbian Deep Learning Without Feedback
• Multiplexed gradient descent: Fast online training of modern datasets on hardware neural networks without backpropagation
• Convolutional Channel-wise Competitive Learning for the Forward-Forward Algorithm
• SymBa: Symmetric Backpropagation-Free Contrastive Learning with Forward-Forward Algorithm for Optimizing Convergence
• The Cascaded Forward Algorithm for Neural Network Training
• Tensor-Compressed Back-Propagation-Free Training for (Physics-Informed) Neural Networks
• Extending the Forward Forward Algorithm
• Graph Neural Networks Go Forward-Forward
• Activation Learning by Local Competitions
• The Trifecta: Three simple techniques for training deeper Forward-Forward networks
• Intrinsic Biologically Plausible Adversarial Robustness
• Emergent representations in networks trained with the Forward-Forward algorithm
• Forward-Forward Algorithm for Hyperspectral Image Classification: A Preliminary Study
• A Study of Forward-Forward Algorithm for Self-Supervised Learning
• Efficient and Flexible Neural Network Training through Layer-wise Feedback Propagation
• A theory for the sparsity emerged in the Forward Forward algorithm
• Layer Collaboration in the Forward-Forward Algorithm
• The Forward-Forward Algorithm as a feature extractor for skin lesion classification: A preliminary study