Models

The basic idea behind all of the NinoLearn models is that they inherit some more general functions from the ninolearn.learn.models.baseModel.baseModel class. In this is done, such that all models can be trained following the same standards. In particular, this is done such that the methods ninolearn.learn.fit.cross_training() and ninolearn.learn.fit.cross_hindcast() work equally for each new model.

Base Model

class ninolearn.learn.models.baseModel.baseModel

The class from which each new model should inherit. Because of the inheritance, standardized training and testing will be possible.

Errors will be raised if mandotory functions are not overwritten by the child model. Mandetory functions are:

1. fit

2. predict

3. save

4. load

fit_RandomizedSearch(trainX, trainy, n_iter=10, **kwargs)

This method performs a random search in the hyperparamter space.

Parameters

• trainX – The feature set.

• trainy – The label set.

• n_iter – The number of iterations for the random search.

• kwargs – Keyword arguments that are passed to the fit method.

set_hyperparameters(**kwargs)

Set the hyperparameters for the model that are provided as keyword arguments.

Parameters

kwargs – The hyperparameters that are used in the child model.

Developed models

Deep Ensemble Model (DEM)

class ninolearn.learn.models.dem.DEM(layers=1, neurons=16, dropout=0.2, noise_in=0.1, noise_mu=0.1, noise_sigma=0.1, noise_alpha=0.1, l1_hidden=0.1, l2_hidden=0.1, l1_mu=0.0, l2_mu=0.1, l1_sigma=0.1, l2_sigma=0.1, l1_alpha=0.1, l2_alpha=0.1, batch_size=10, n_segments=5, n_members_segment=1, lr=0.001, patience=10, epochs=300, verbose=0, pdf='normal', name='dem')

A deep ensemble model (DEM) predicting either mean or mean and standard deviation with one hidden layer having the ReLU function as activation for the hidden layer. It is trained using the MSE or negative-log-likelihood of a gaussian distribution, respectively.

Parameters

• layers (int) – Number of hidden layers.

• neurons (int) – Number of neurons in a hidden layers.

• dropout (float) – Dropout rate for the hidden layer neurons.

• noise (float) – Standard deviation of the gaussian noise that is added to the input

• l1_hidden (float) – Coefficent for the L1 penalty term for the hidden layer.

• l2_hidden (float) – Coefficent for the L2 penalty term for the hidden layer.

• l1_mu (float) – Coefficent for the L1 penalty term in the mean-output neuron.

• l2_mu (float) – Coefficent for the L2 penalty term in the mean-output neuron.

• l1_sigma (float) – Coefficent for the L1 penalty term in the standard-deviation-output neuron.

• l2_mu – Coefficent for the L2 penalty term in the standard-deviation output neuron.

• batch_size – Batch size for the training.

• n_segments – Number of segments for the generation of members.

• n_members_segment – number of members that are generated per segment.

• lr – the learning rate during training

• patience – Number of epochs to wait until training is stopped if score was not improved.

• epochs – The maximum numberof epochs for the training.

• verbose – Option to print scores during training to the screen. Here, 0 means silent.

• pdf (str) – The distribution which shell be predicted. Either ‘simple’ (just one value), ‘normal’ (Gaussian) or ‘skewed’ (skewed Gaussian).

• name (str) – The name of the model.

build_model(n_features)

The method builds a new member of the ensemble and returns it.

evaluate(ytrue, mean_pred, std_pred=False)

Negative - log -likelihood for the prediction of a gaussian probability

fit(trainX, trainy, valX=None, valy=None, use_pretrained=False)

Fit the model to training data

get_model_desc(pdf)

Assignes sum weights description to the model depending on which predicted distribution is selected.

load(location=None, dir_name='dem')

Load the ensemble

predict(X)

Generates the ensemble prediction of a model ensemble

Parameters

• model_ens – list of ensemble models

• X – The features

save(location='', dir_name='ensemble')

Save the ensemble

save_weights(location='', dir_name='ensemble')

Saves the weights

Encoder-Decoder (ED)

class ninolearn.learn.models.encoderDecoder.EncoderDecoder

The Encoder-Decoder is an neural network that has the same architecture as an Autoencoder. Hence, labal and feature vector have the same dimension. In the ninolearn package the model is called Encoder-Decoder because it is used for prediction purposes and therefore label and feature vector might be soparated by some time lag or even are not the same variable.

build_model(n_features, n_labels)

The method builds a new member of the ensemble and returns it.

Parameters

• n_features (int) – The number of features.

• n_labels (int) – The number of labels.

evaluate(X, ytrue)

Evaluate the model based on the RMSE

Parameters

• X (np.ndarray) – The feature array.

• ytrue (np.ndarray) – The true label array.

fit(trainX, trainy, valX=None, valy=None)

Fit the model. If n_segments is 1, then a validation data set needs to be supplied.

Parameters

• trainX (np.ndarray) – The training feature set. 2-D array with dimensions (timesteps, features)

• trainy (np.ndarray) – The training label set. 2-D array with dimensions (timesteps, labels)

• valX (np.ndarray) – The validation feature set. 2-D array with dimensions (timesteps, features).

• valy (np.ndarray) – The validation label set. 2-D array with dimensions (timesteps, labels).

fit_RandomizedSearch(trainX, trainy, n_iter=10, **kwargs)

Hyperparameter optimazation using random search.

Parameters

• trainX (np.ndarray) – The training feature set. 2-D array with dimensions (timesteps, features).

• trainy (np.ndarray) – The training label set. 2-D array with dimensions (timesteps, labels).

• kwargs – Keyword arguments are passed to the .fit() method.

load(location=None, dir_name='ensemble')

Save the ensemble.

Parameters

• location (str) – Base directory where for all Encoder-Decoder ensembles.

• dir_name (str) – The specific directory name in the base directory were to find the ensemble.

predict(X)

Ensemble prediction.

Parameters

X (np.ndarray) – Feature set for which the prediction should be made.

save(location='', dir_name='ed_ensemble')

Save the ensemble.

Parameters

• location (str) – Base directory where to for all Encoder-Decoder ensembles

• dir_name (str) – The specific directory name in the base directory were to save the ensemble.

set_parameters(neurons=(128, 16), dropout=0.2, noise=0.2, noise_out=0.2, l1_hidden=0.0001, l2_hidden=0.0001, l1_out=0.0001, l2_out=0.0001, batch_size=50, lr=0.0001, n_segments=5, n_members_segment=1, patience=40, epochs=500, verbose=0)

Set the parameters of the Encoder-Decoder neural network.

Note, if the parameters are given in a list, ninolearn assumes that a the method .fit_RandomizedSearch() is used.

Parameters

• neurons (tuple (list of two tuples for .fit_RandomizedSearch())) – The architecture of the Encoder-Decoder. The layer with the lowest number of neurons is assumed to be the bottleneck layer for which the activation function is linear. Furthermore, the output layer has a linear activation as well. All other layers have the ReLU as activation.

• dropout (float) – Standard deviation of the Gaussian dropout. Dropout layers are installed behind each hidden layer in the Encoder and the Decoder.

• noise (float) – Standard deviation of Gaussian noise for the input layer.

• noise – Standard deviation of Gaussian noise for the output layer.

• l2_hidden (l1_hidden,) – Coefficent for the L1 and the L2 penalty term for the hidden layer weights.

• l2_hidden – Coefficent for the L1 and the L2 penalty term for the output layer weights.

• batch_size (int) – Batch size during training of a member of the Encoder-Decoder.

• lr (float) – The learning rate.

• n_segments (int) – The number of segments that are used for the cross- validation scheme and the training of the Ensemble members.

• n_members_segment (int) – The number of members that are trained for one segment.

• patience (int) – The number of epochs to wait until Early-Stopping stops the training.

• epochs (int) – The maximum number of epochs.

• verbose (int) – Print some progress to screen. Either 0 (silent), 1 or 2.