Fractional LMS (FLMS)¶
Implements Fractional LMS (FLMS), a least-mean-squares filter whose update augments the conventional gradient with a Caputo fractional-derivative term.
FLMS minimizes the instantaneous mean-square error \(e_t = d_t - \theta_t^\top x_t\) by adapting the filter weights along both the integer-order gradient and a fractional-order gradient of the same cost. For the quadratic cost the fractional (\(\alpha\)-order) derivative with respect to a weight contributes a factor \(|\theta_t|^{1-\alpha}/\Gamma(2-\alpha)\), so the fractional step scales each coordinate by the magnitude of its current weight raised to \(1-\alpha\). The absolute value is the remedy this analysis adopts to prevent the complex-valued iterates that arise when raising negative weights to a fractional power.
where \(\theta_t^{(i)}\) is the \(i\)-th filter weight, \(x_t\) the input regressor, \(d_t\) the desired output, \(e_t\) the error, \(\gamma>0\) the conventional learning rate, \(\gamma_f\) the fractional learning rate, \(\alpha\in(0,1)\) the fractional order, and \(\Gamma\) the Euler gamma function. (The term \(e_t x_t^{(i)}\) is the negative gradient of the squared error, so each line is a gradient-ascent-on-\(-e_t^2\) step.)
Reference: Abdul Wahab, Shujaat Khan, Imran Naseem, Jong Chul Ye, "Performance Analysis of Fractional Learning Algorithms", arXiv 2021. https://arxiv.org/abs/2110.05201