Design of generalized fractional order gradient descent method

Abstract

This paper focuses on the convergence problem of the emerging fractional order gradient descent method, and proposes three solutions to overcome the problem. In fact, the general fractional gradient method cannot converge to the real extreme point of the target function, which critically hampers the application of this method. Because of the long memory characteristics of fractional derivative, fixed memory principle is a prior choice. Apart from the truncation of memory length, two new methods are developed to reach the convergence. The one is the truncation of the infinite series, and the other is the modification of the constant fractional order. Finally, six illustrative examples are performed to illustrate the effectiveness and practicability of proposed methods.

8 pages, 16 figures

References in corpus (2)

• Convolutional neural networks with fractional order gradient method
• A Dynamical Systems Perspective on Nesterov Acceleration

Cited by in corpus (7)

• Performance Analysis of Fractional Learning Algorithms
• A cookbook for finite element methods for nonlocal problems, including quadrature rules and approximate Euclidean balls
• A Caputo fractional derivative-based algorithm for optimization
• An overview of the fractional-order gradient descent method and its applications
• Sequential bi-level regularized inversion with application to hidden reaction law discovery
• Comments on "Generalization of the gradient method with fractional order gradient direction"
• Comments on "A new computing approach for power signal modeling using fractional adaptive algorithms"