Logistic Regression#

A linear classifier that uses the logistic (sigmoid) function to estimate the probabilities of exactly two class outcomes. The model parameters (weights and bias) are solved using Mini Batch Gradient Descent with pluggable optimizers and cost functions that run on the neural network subsystem. In addition, the learner features progress monitoring which stops training when it can no longer improve the validation score. It also utilizes network snapshotting to make sure that it always has the best model parameters even if progress began to decline during training.

Note

Progress monitoring and early stopping require a validation set. Use setValidationDataset() to supply one.

Interfaces: Estimator, Learner, Online, Probabilistic, Ranks Features, Verbose, Persistable

Data Type Compatibility: Continuous

Parameters#

#	Name	Default	Type	Description
1	batchSize	128	int	The number of training samples to process at a time.
2	optimizer	Adam	Optimizer	The gradient descent optimizer used to update the network parameters.
3	l1Penalty	1e-4	float	The amount of L1 regularization applied to the weights of the output layer.
4	l2Penalty	1e-4	float	The amount of L2 regularization applied to the weights of the output layer.
5	epochs	1000	int	The maximum number of training epochs. i.e. the number of times to iterate over the entire training set before terminating.
6	minChange	1e-5	float	The minimum change in the training loss necessary to continue training.
7	evalInterval	1	int	The number of epochs to train before evaluating the model using the validation set.
8	window	10	int	The number of evaluations without improvement in the validation score to wait before considering an early stop. Set to 0 to disable early stopping.
9	costFn	BinaryCrossEntropy	ClassificationLoss	The function that computes the loss associated with an erroneous activation during training.
10	metric	FBeta	Metric	The validation metric used to score the generalization performance of the model during training.

Example#

use Rubix\ML\Classifiers\LogisticRegression;
use Rubix\ML\NeuralNet\Optimizers\Adam;
use Rubix\ML\NeuralNet\Optimizers\Schedulers\Constant;
use Rubix\ML\NeuralNet\CostFunctions\BinaryCrossEntropy;
use Rubix\ML\CrossValidation\Metrics\MCC;

$estimator = new LogisticRegression(
    batchSize: 64,
    optimizer: new Adam(scheduler: new Constant(0.001)),
    l1Penalty: 1e-4,
    l2Penalty: 1e-4,
    epochs: 100,
    minChange: 1e-5,
    evalInterval: 1,
    window: 10,
    costFn: new BinaryCrossEntropy(),
    metric: new MCC()
);

Additional Methods#

Return the loss for each epoch from the last training session.

public losses() : float[]|null

Return the progress table combining every epoch recorded during the last training session — the loss, the validation score, and the gradient norm when available — into a single ordered sequence.

public progress() : iterable

Set the dataset used to score the model during training. Once a validation dataset is set, evalInterval and window determine how often it is scored and when training stops early. Pass null to disable progress monitoring and early stopping.

public setValidationDataset(?Labeled $dataset) : void

Return the validation score for each epoch from the last training session.

public scores() : float[]|null

Returns the underlying neural network instance or null if untrained. See FeedForward for more details.

public network() : ?\Rubix\ML\NeuralNet\Network

Set the path of the temporary snapshot file used to store network parameters during training.

public setSnapshotPath(?string $path) : void

Clean up any leftover state after training. Only do this if you plan to use the model for inference.

public cleanup() : void