NGLFGD¶

Implements NGLFGD, a non-causal Grünwald–Letnikov fractional-order gradient descent.

The Grünwald–Letnikov fractional derivative of order \(\alpha\) is one-sided: the backward (causal) form looks only at past samples and the forward (anti-causal) form only at future ones, and each carries a phase shift that biases the descent direction. NGLFGD builds a non-causal fractional derivative by subtracting the anti-causal series from the causal one, so the imaginary part of the phase spectrum cancels and the operator shares the same phase as the ordinary first derivative while keeping the long memory of the fractional order. Per coordinate \(i\), the loss is sampled at integer offsets \(j\) along \(e_i\) on both sides, and the truncated Grünwald–Letnikov binomial weights combine those samples into a fractional gradient, which then drives a plain gradient-descent step.

\[ \begin{aligned} c_j &= (-1)^j \binom{\alpha}{j} = \frac{(-1)^j}{\Gamma(j+1)}\prod_{i=0}^{j-1}(\alpha - i), \\ g_t^{(i)} &= \sum_{j=0}^{k-1} c_j \left[\, f(\theta_t - j\,e_i) - f(\theta_t + j\,e_i) \,\right], \\ \theta_{t+1} &= \theta_t - \eta\, g_t. \end{aligned} \]

where \(\alpha \in (0,1]\) is the fractional order, \(k\) is the truncation length of the Grünwald–Letnikov expansion, \(c_j\) are its binomial coefficients, \(e_i\) is the \(i\)-th unit vector, \(f\) is the objective, \(g_t\) is the assembled non-causal fractional gradient (causal minus anti-causal), and \(\eta\) is the learning rate.

Reference: Haixin Wu, Jiacheng Weng, Yuanzheng Lou, Yuqiu Shen, Jianhong Wang, "Fast and accurate fractional order gradient descent algorithm and its application in Extreme Gradient Boosting", Knowledge-Based Systems 2026. https://doi.org/10.1016/j.knosys.2025.114911

Back to the Canon