Fractional-Order Optimizers

Fractional-order optimizers generalize the integer-order gradient step with fractional-calculus operators, most commonly the Caputo, Riemann-Liouville, or Grünwald-Letnikov derivative, which weight past gradient information through power-law memory kernels. The field is young: the first neural-network training results date to 2015, convergence theory is still being settled, and most papers ship no code.

Foundations

Optimizer	Venue	Paper	Code
the Fractional Steepest Descent Method (FSDM)	IEEE Transactions on Neural Networks and Learning Systems 2015	Fractional Extreme Value Adaptive Training Method: Fractional Steepest Descent Approach	—
Caputo BP-NN FOGD (ISNN)	Advances in Neural Networks - ISNN 2017 (Lecture Notes in Computer Science)	A Caputo-Type Fractional-Order Gradient Descent Learning of BP Neural Networks	—
Caputo CVNN FOGD	IEEE Access 2017	Convergence Analysis of Caputo-Type Fractional Order Complex-Valued Neural Networks	—
Caputo fractional-order gradient descent	Neural Networks 2017	Fractional-order gradient descent learning of BP neural networks with Caputo derivative	—
FBPTT	Circuits, Systems, and Signal Processing 2018	A Novel Fractional Gradient-Based Learning Algorithm for Recurrent Neural Networks	—
FGD-RBF	Circuits, Systems, and Signal Processing 2018	A Fractional Gradient Descent-Based RBF Neural Network	—
Fractional-Order Deep BP NN	Computational Intelligence and Neuroscience 2018	Fractional-Order Deep Backpropagation Neural Network	official
Caputo-Type FOGD (Deep BP)	IEEE IMCEC 2019	A Caputo-Type Fractional-Order Gradient Descent Learning of Deep BP Neural Networks	—
FSGD	Electronic Markets 2019	Fractional stochastic gradient descent for recommender systems	—
mF-SGD	IEEE Access 2019	Design of Momentum Fractional Stochastic Gradient Descent for Recommender Systems	—
CFEM-LMS	Neurocomputing 2020	Combination of fractional FLANN filters for solving the Van der Pol-Duffing oscillator	—
FSDM	Frontiers of Information Technology & Electronic Engineering 2020	Fractional-order global optimal backpropagation machine trained by an improved fractional-order steepest descent method	—
Fractional Order Gradient Method	Neurocomputing 2020	Convolutional neural networks with fractional order gradient method	—
Normalized Fractional SGD (NFSGD)	Neural Computing and Applications 2020	Design of normalized fractional SGD computing paradigm for recommender systems	—
the Fractional Order Gradient Method	Journal of the Franklin Institute 2020	Generalization of the gradient method with fractional order gradient direction	—
Fractional Order Gradient Descent with Momentum (FOGDM)	Network: Computation in Neural Systems 2020	Data classification based on fractional order gradient descent with momentum for RBF neural network	—
CFGD (Caputo)	arXiv 2021	A Caputo fractional derivative-based algorithm for optimization	—
Fractional-Order Momentum (FCM)	Neurocomputing 2021	Convolutional neural networks based on fractional-order momentum for parameter training	—
FOGDM-RBF	Soft Computing 2021	Fractional-order gradient descent with momentum for RBF neural network-based AIS trajectory restoration	—
Caputron	Electronics (MDPI) 2022	Exploring the Effects of Caputo Fractional Derivative in Spiking Neural Network Training	official
FGD (CNN BP)	arXiv 2022	Using a novel fractional-order gradient method for CNN back-propagation	—
FGNN	Mathematics (MDPI) 2022	A Regularized Graph Neural Network Based on Approximate Fractional Order Gradients	—
FracM	Neural Computing and Applications 2022	A fractional-order momentum optimization approach of deep neural networks	community
GFSGD	Chaos, Solitons & Fractals 2022	Generalized fractional strategy for recommender systems with chaotic ratings behavior	—
Fractional Derivative Gradient Optimizers (FSGD	Applied Sciences 2022	Fractional Derivative Gradient-Based Optimizers for Neural Networks and Human Activity Recognition	—
Fractional LMS (FLMS)	IEEE Transactions on Signal Processing 2022	Performance Analysis of Fractional Learning Algorithms	—
Conformable Fractional Gradient Descent	Fuzzy Systems and Data Mining VIII 2022	Fractional Gradient Descent Learning of Backpropagation Artificial Neural Networks with Conformable Fractional Calculus	—
Fractional Order Gradient Descent with variable initial value	Neurocomputing 2022	Study on fast speed fractional order gradient descent method and its application in neural networks	—
TFGD (Time-fractional)	Axioms 2022	Training Neural Networks by Time-Fractional Gradient Descent	—
Variable Order Fractional Gradient Descent	Chinese Control and Decision Conference 2022	Variable Order Fractional Gradient Descent Method and Its Application in Neural Networks Optimization	—
CfGD / CfAdam	Neural Networks 2023	Accelerating gradient descent and Adam via fractional gradients	—
RFGD	Neural Networks 2023	A fractional gradient descent algorithm robust to the initial weights of multilayer perceptron	—
FO-RI-FedAvg	arXiv 2026	Fractional Order Federated Learning for Battery Electric Vehicle Energy Consumption Modeling	—
IHL-Adam	Expert Systems with Applications 2024	Parameter training method for convolutional neural networks based on improved Hausdorff-like derivative	—

Recent advances

Optimizer	Venue	Paper	Code
AFOGD / AFOAGD	arXiv 2023	The Novel Adaptive Fractional Order Gradient Decent Algorithms Design via Robust Control	—
EFSGD / EN-EFSGD	Chaos, Solitons & Fractals 2023	Enhanced fractional prediction scheme for effective matrix factorization in chaotic feedback recommender systems	—
FCGD_G-L	Mathematics 2023	A Deep Learning Optimizer Based on Grünwald–Letnikov Fractional Order Definition	—
FGDAM	Applied Mathematics and Computation 2023	Applications of fractional gradient descent method with adaptive momentum in BP neural networks	—
FracG	Chinese Control Conference (CCC) 2023	Optimization Method of Neural Networks via Fractional-Order of Gradients	—
Fractional Gradient Descent (FSGD)	Fractal and Fractional 2023	Fractional Gradient Optimizers for PyTorch: Enhancing GAN and BERT	—
the Improved Stochastic Fractional Order Gradient Descent algorithm	Fractal and Fractional 2023	The Improved Stochastic Fractional Order Gradient Descent Algorithm	—
AdaGL	Neural Processing Letters 2024	An Adaptive Learning Rate Deep Learning Optimizer Using Long and Short-Term Gradients Based on G–L Fractional-Order Derivative	community
GFSGD	Heliyon 2024	Fractional gradient optimized explainable convolutional neural network for Alzheimer's disease diagnosis	—
FOAdam	Applied Mathematical Modelling 2024	A novel gradient descent optimizer based on fractional order scheduler and its application in deep neural networks	—
Adaptive Terminal Caputo Fractional Gradient Descent (AT-CFGD)	TMLR 2024	Convergence Analysis of Fractional Gradient Descent	—
Caputo Fractional-Order Gradient Descent	International Journal of Fuzzy Systems 2024	A Novel Neuro-fuzzy Learning Algorithm for First-Order Takagi–Sugeno Fuzzy Model: Caputo Fractional-Order Gradient Descent Method	—
FNGD	IEEE Access 2024	Improving the Accuracy of Neural Network Pattern Recognition by Fractional Gradient Descent	—
MFFGD	Neurocomputing 2024	MFFGD: An adaptive Caputo fractional-order gradient algorithm for DNN	—
Caputo-based SGD (L1 scheme)	OpenReview 2024	Stochastic Fractional Gradient Descent with Caputo L1 Scheme for Deep Neural Networks	—
C-FOG	Fractal and Fractional 2024	Self-Organizing Optimization Based on Caputo's Fractional Order Gradients	—
CSA-CFGD	PeerJ Computer Science 2024	Deep ocular tumor classification model using cuckoo search algorithm and Caputo fractional gradient descent	official
FGD-RBFNN (UAV)	Computer Modeling in Engineering & Sciences 2024	Fractional Gradient Descent RBFNN for Active Fault-Tolerant Control of Plant Protection UAVs	—
FOELM	Applied Soft Computing 2024	An interval neural network-based Caputo fractional-order extreme learning machine applied to classification	—
MIF	Algorithms 2024	An Integer-Fractional Gradient Algorithm for Back Propagation Neural Networks	—
Multi-layer NN FOGD	Advanced Theory and Simulations 2024	Convergence Analysis and Application for Multi-Layer Neural Network Based on Fractional-Order Gradient Descent Learning	—
UCAdam	Journal of Electrical Systems 2024	Improved Adam: Incorporating Unified Conformable Fractional Derivative for fractional-order Momentum	—
2SEDFOSGD	arXiv 2025	Effective Dimension Aware Fractional-Order Stochastic Gradient Descent for Convex Optimization Problems	—
2SEDFOSGD	arXiv 2025	More Optimal Fractional-Order Stochastic Gradient Descent for Non-Convex Optimization Problems	—
AFGD (adaptive Caputo FGD for TCN)	Neurocomputing 2025	Monotonic convergence of adaptive Caputo fractional gradient descent for temporal convolutional networks	—
FGDSINN	International Journal of Machine Learning and Cybernetics 2025	A smoothing interval neural networks-based Caputo fractional-order gradient learning algorithm	—
FOSGD / FOSGDM / FOSGDME	Neural Networks 2025	Fractional-order stochastic gradient descent method with momentum and energy for deep neural networks	—
FracGrad	Fractal and Fractional 2025	FracGrad: A Discretized Riemann–Liouville Fractional Integral Approach to Gradient Accumulation for Deep Learning	—
GF-SGD	Computers in Biology and Medicine 2025	Generalized fractional optimization-based explainable lightweight CNN model for malaria disease classification	—
IFOGD	Neural Networks 2025	Improved fractional-order gradient descent method based on multilayer perceptron	—
L2O-CFGD	arXiv 2025	Enhancing Fractional Gradient Descent with Learned Optimizers	official
MOAOCFGD	arXiv 2025	An Adaptive Order Caputo Fractional Gradient Descent Method for Multi-objective Optimization Problems	—
NCFDD / NFLightGBM	Information Fusion 2025	Fractional light gradient boosting machine ensemble learning model: A non-causal fractional difference descent approach	—
a Caputo fractional-order gradient descent for neural network training	Chaos, Solitons & Fractals 2025	Fractional-order gradient approach for optimizing neural networks: A theoretical and empirical analysis	—
Fractional-order SGD (FSGD)	arXiv 2025	Fractional-order Jacobian Matrix Differentiation and Its Application in Artificial Neural Networks	—
Adaptive Parameter Fractional-Order Gradient Descent Learning	European Journal of Operational Research 2025	Novel adaptive parameter fractional-order gradient descent learning for stock selection decision support systems	—
FAdam	Chaos, Solitons & Fractals 2025	Parameter training methods for convolutional neural networks with adaptive adjustment method based on Caputo fractional-order differences	—
SFM	Digital Signal Processing 2025	A momentum-based stochastic fractional gradient optimizer with U-net model for brain tumor segmentation in MRI	—
Caputo Fractional-order Gradient Descent for Ridge Polynomial Neural	International Conference on Electronics and Communication, Network and Computer Technology 2025	A Novel Method for Ridge Polynomial Neural Network-based Caputo Fractional-order Gradient Descent Algorithm	—
AOFGD	SSRN 2025	AOFGD: Adaptive order fractional gradient descent method	—
Frac-Adam	Mathematics 2025	Fractional Optimizers for LSTM Networks in Financial Time Series Forecasting	—
Caputo Fractional Gradient Descent	International Conference on Advanced Algorithms and Control Engineering 2025	Fractional Order Gradient Descent with Caputo Derivatives for Product-Unit Neural Networks	—
FO-STDGD	Neurocomputing 2025	Fractional-order spike-timing-dependent gradient descent for multi-layer spiking neural networks	—
Fractional Order Stochastic Gradient Descent (FOSGD)	ASME IDETC-CIE 2025	Tail-Index-Awareness in Fractional Order Stochastic Gradient Descent	—
λ-FAdaMax	Expert Systems with Applications 2025	λ-FAdaMax: A novel fractional-order gradient descent method with decaying second moment for neural network training	—
CFDNN	Scientific Reports 2026	Conformable Fractional Deep Neural Networks (CFDNN) for high-speed cyber-attack detection	—
CFGD (Compressed)	IEEE Transactions on Neural Networks and Learning Systems 2026	Fractional Gradient Descent With Matrix Stepsizes for Non-Convex Optimization	official
FAdamWav	Fractal and Fractional 2026	FAdamWav: A Fractional Wavelet Gradient Optimizer for Neural Networks	—
FOFedAvg	arXiv 2026	Fractional-Order Federated Learning	—
Fractional-order FL with adaptive momentum	IEEE Transactions on Emerging Topics in Computational Intelligence 2026	Communication-Efficient Federated Learning via Fractional-Order Gradient Descent With Adaptive Momentum Under Non-IID Data	—
TFGD (Tempered)	Neural Networks 2026	Tempered fractional gradient descent: Theory, algorithms, and robust learning applications	—
FGD-ED	Information Processing & Management 2026	Fractional-order gradient descent method based on fractional-order term exponential decay and its application in artificial neural networks	—
the Caputo Fractional-Order Gradient Descent Method (FGDM)	Applied Soft Computing 2026	A novel gradient learning algorithm based on zero-order Takagi-Sugeno fuzzy model: the caputo fractional-order gradient descent	—
CFGD (Conformable)	Journal of Computational and Applied Mathematics 2026	Conformable fractional gradient descent: A local optimizer for neural network training	—
NGLFGD	Knowledge-Based Systems 2026	Fast and accurate fractional order gradient descent algorithm and its application in Extreme Gradient Boosting	—
FO-Elman	Neural Networks 2026	Fractional-order gradient descent learning for Elman neural networks	—

Surveys

Optimizer	Venue	Paper	Code
Fractional-Order Gradient Descent for Neural Networks	The European Physical Journal Special Topics 2022	Artificial neural networks: a practical review of applications involving fractional calculus	—
Fractional Gradient Descent (FGD)	Chaos, Solitons & Fractals 2025	A comprehensive survey of fractional gradient descent methods and their convergence analysis	—
the Fractional Continuous Time Method (FCTM)	Journal of Computational and Applied Mathematics 2026	An overview of the fractional-order gradient descent method and its applications	—

Note: FAdam (arXiv 2405.12807) is a Fisher-information variant of Adam and is unrelated to fractional calculus despite the name.