[source]

Feed Forward

The Feed Forward network is the core neural network implementation of the library consisting of an input layer, any number of intermediate hidden layers, and an output layer. The parameters of the network are learned using mini batch gradient descent with backpropagation. It is the network used under the hood by the neural network learners such as Multilayer Perceptron, MLP Regressor, Adaline, Softmax Classifier, and Logistic Regression.

Note

The Feed Forward network is part of the neural network subsystem and is not a standalone estimator.

Parameters

#	Name	Default	Type	Description
1	input		Input	The input layer.
2	hidden		array	The array of hidden layers in the network.
3	output		Output	The output layer.

Example

use Rubix\ML\NeuralNet\FeedForward;
use Rubix\ML\NeuralNet\Layers\Placeholder1D;
use Rubix\ML\NeuralNet\Layers\Dense;
use Rubix\ML\NeuralNet\Layers\Activation;
use Rubix\ML\NeuralNet\Layers\Multiclass;
use Rubix\ML\NeuralNet\ActivationFunctions\LeakyReLU;
use Rubix\ML\NeuralNet\CostFunctions\MulticlassCrossEntropy;

$network = new FeedForward(
    new Placeholder1D(784),
    [
        new Dense(neurons: 200),
        new Activation(activationFn: new LeakyReLU()),
        new Dense(neurons: 100),
        new Activation(activationFn: new LeakyReLU()),
    ],
    new Multiclass(
        numClasses: 3,
        costFn: new MulticlassCrossEntropy()
    )
);

API Reference

Return the input layer of the network:

public function input() : Input

Return an array of hidden layers indexed left to right:

public function hidden() : array

Return the output layer of the network:

public function output() : Output

Return all the layers in the network in the order they are executed:

public function layers() : Traversable

Return the total number of parameters in the network:

public function numParams() : int

Return an iterable of all the parameters in the network:

public function parameters() : Traversable

Return the number of trainable (unfrozen) parameters in the network:

public function numTrainableParams() : int

Return an iterable of all the trainable (unfrozen) parameters in the network:

public function trainableParameters() : Traversable

Initialize the parameters of the layers. Called once before the first training session to set up the network:

public function initialize() : void

Freeze the first k hidden layers of the network preventing their parameters from being updated during training. Useful for fine-tuning a pretrained model.

public function freezeFirstKLayers(int $k) : void

Unfreeze all hidden layers allowing their parameters to be updated during training.

public function unfreeze() : void

Run an inference pass and return the activations at the output layer:

public function infer(Matrix $x) : Matrix

Note

Output layers expect labels as a 2-dimensional target matrix in which the outer dimension indexes the output targets and the inner dimension indexes the samples in the batch. For example, a binary classification layer expects a single inner array of 0/1 labels whereas a Multiclass layer expects a one-hot encoded matrix with one row per class. The learners build the target matrix from the labels of the batch and feed it to the network — for instance, the Multilayer Perceptron and Softmax Classifier expand the class indices of each sample into a one-hot encoded matrix before backpropagating.

Feed a batch through the network and return a matrix of activations at the output layer:

public function feed(Matrix $x) : Matrix

Backpropagate the gradient of the cost function and return the loss:

public function backpropagate(Matrix $y) : float

The backpropagate method accepts the same target matrix as the output layers.

Export the network architecture as a graph in dot format:

public function exportGraphviz() : Encoding

use Rubix\ML\Helpers\Graphviz;
use Rubix\ML\Persisters\Filesystem;

$dot = $network->exportGraphviz();

Graphviz::dotToImage($dot)->saveTo(new Filesystem('network.png'));