the Fractional Continuous Time Method (FCTM)¶
Implements the Fractional Continuous Time Method (FCTM), gradient descent driven by a fractional-order time derivative.
Standard gradient descent is the explicit Euler discretization of the gradient flow \(\dot{u} = -\lambda f'(u)\). Fractional gradient descent methods instead replace the spatial gradient with a Caputo (or Riemann-Liouville) fractional derivative \(D_u^{\alpha}\), giving the closed-form update \(u_{k+1} = u_k - \omega\,[D_u^{\alpha} f](u_k)\); the drawback is that its fixed points satisfy \(D_u^{\alpha} f = 0\) rather than \(f'(u) = 0\), so it need not converge to the true extremum. FCTM keeps the ordinary gradient but replaces the integer time derivative with a Caputo fractional derivative of order \(\alpha\), so that equilibria still coincide with the stationary points of \(f\) while the dynamics gain long memory. The resulting fractional ODE is integrated numerically with an Adams-Bashforth-Moulton predictor-corrector (PECE) scheme.
where \(u\) are the parameters, \(f\) the objective, \(\lambda\) and \(\omega\) the learning rate / step size, \(\alpha\) the fractional order (convergence proven for \(0 < \alpha \le 1\), with simulations suggesting \(1 \le \alpha \le 2\)), \(\Gamma\) the gamma function, \(a\) the lower terminal of the fractional operator, and \({}^{*}D\) the Caputo derivative. The first line is FCTM's continuous formulation (solved via PECE); the last line is the closed-form fractional gradient descent update it reviews.
Reference: Higor V. M. Ferreira, Camila A. Tavares, Nelson H. T. Lemes, José P. C. dos Santos, "An overview of the fractional-order gradient descent method and its applications", arXiv 2026. https://arxiv.org/abs/2601.03318