[source]

RMS Prop

An adaptive gradient technique that divides the current gradient over a rolling window of the magnitudes of recent gradients. Unlike AdaGrad, RMS Prop does not suffer from an infinitely decaying step size.

Parameters

#	Name	Default	Type	Description
1	scheduler		Scheduler	The learning-rate schedule that supplies the step size each batch.
2	decay	0.1	float	The decay rate of the rms property.

Example

use Rubix\ML\NeuralNet\Optimizers\RMSProp;
use Rubix\ML\NeuralNet\Optimizers\Schedulers\Constant;

$optimizer = new RMSProp(scheduler: new Constant(0.01), decay: 0.1);

References

---

1. T. Tieleman et al. (2012). Lecture 6e rmsprop: Divide the gradient by a running average of its recent magnitude. ↩