NNInterface

class ketos.neural_networks.dev_utils.nn_interface.NNInterface(optimizer, loss_function, metrics)

General interface for neural network architectures in the ketos.neural_networks module.

This class implements common methods for neural network models and is supposed to be subclassed. When implementing new neural network architectures, the interface implemented in this class can be inherited.

Args:

optimizer: RecipeCompat object

An instance of the RecipeCompat class wrapping a tensorflow(-compatible) optimizer (e.g.:from tensorflow.keras.optimizers)

loss_function: RecipeCompat object

An instance of the RecipeCompat class wrappinf a tensorflow(-compatible) loss-function (e.g.:from tensorflow.keras.losses)

metrics: list of RecipeCompat objects

A list of instances of the RecipeCompat class wrapping a tensorflow(-compatible) metric (e.g.:from tensorflow.keras.metrics)

Examples:

The following example shows how a newly defined network architecture could use the interface provided by NNInterface.

First, the new architecture must be defined. Here, a simple multi-layer perceptron is defined in the following class.

>>> class MLP(tf.keras.Model): 
...    def __init__(self, n_neurons, activation):
...        super(MLP, self).__init__()
... 
...        self.dense = tf.keras.layers.Dense(n_neurons, activation=activation)
...        self.final_node = tf.keras.layers.Dense(1)
... 
...    def call(self, inputs):
...        output = self.dense(inputs)
...        output = self.dense(output)
...        output = self.final_node(output)
...        return output

With the architecture, the interface to the MLP can be created by subclassing NNInterface:

>>> from ketos.neural_networks.dev_utils import RecipeCompat, NNInterface   
>>> class MLPInterface(NNInterface):  
...     @classmethod
...     def _build_from_recipe(cls, recipe, recipe_compat=True):
...         n_neurons = recipe['n_neurons']    # take the n_neurons parameter from the recipe instead of using the default
...         activation = recipe['activation']  # take the activation parameter from the recipe instead of using the default
...        
...         if recipe_compat == True:
...             optimizer = recipe['optimizer']
...             loss_function = recipe['loss_function']
...             metrics = recipe['metrics']
...            
...         else:
...             optimizer = cls._optimizer_from_recipe(recipe['optimizer'])
...             loss_function = cls._loss_function_from_recipe(recipe['loss_function'])
...             metrics = cls._metrics_from_recipe(recipe['metrics'])
...
...         instance = cls(n_neurons=n_neurons, activation=activation, optimizer=optimizer, loss_function=loss_function, metrics=metrics)
...         return instance
... 
...     @classmethod
...     def _read_recipe_file(cls, json_file, return_recipe_compat=True):
...         with open(json_file, 'r') as json_recipe:
...            recipe_dict = json.load(json_recipe)
...
...         optimizer = cls.optimizer_from_recipe(recipe_dict['optimizer'])
...         loss_function = cls.loss_function_from_recipe(recipe_dict['loss_function'])
...         metrics = cls.metrics_from_recipe(recipe_dict['metrics'])
...
...         if return_recipe_compat == True:
...             recipe_dict['optimizer'] = optimizer
...             recipe_dict['loss_function'] = loss_function
...             recipe_dict['metrics'] = metrics
...         else:
...             recipe_dict['optimizer'] = cls._optimizer_to_recipe(optimizer)
...             recipe_dict['loss_function'] = cls._loss_function_to_recipe(loss_function)
...             recipe_dict['metrics'] = cls._metrics_to_recipe(metrics)
...
...        recipe_dict['n_neurons'] = recipe_dict['n_neurons']    # read the n_neurons parameter from the recipe file
...        recipe_dict['activation'] = recipe_dict['activation']  # read the activation parameter from the recipe file
...        
...        return recipe_dict
...
...     def __init__(self, n_neurons, activation, optimizer, loss_function, metrics):
...        super(MLPInterface, self).__init__(optimizer, loss_function, metrics)
...        self.n_neurons = n_neurons
...        self.activation = activation
...        self.model = MLP(n_neurons=n_neurons, activation=activation)
...
...    def _extract_recipe_dict(self):
...        recipe = {}
...        recipe['optimizer'] = self._optimizer_to_recipe(self.optimizer)
...        recipe['loss_function'] = self._loss_function_to_recipe(self.loss_function)
...        recipe['metrics'] = self._metrics_to_recipe(self.metrics)
...        recipe['n_neurons'] = self.n_neurons
...        recipe['activation'] = self.activation
...        return recipe

Methods

add_learning_rate_scheduler([scheduler_type])	Add a learning rate scheduler to the current neural network interface.
build(recipe_file)	Build a model from a recipe file
load(model_file[, new_model_folder, ...])	Load a model from a ketos (.kt) model file.
run_on_batch(input_batch[, ...])	Run the model on a batch of inputs
run_on_instance(input[, return_raw_output, ...])	Run the model on one input
run_on_test_generator([return_raw_output, ...])	Run the model on the test generator
save([output_name, checkpoint_name])	Save the trained model to a file.
save_recipe_file(recipe_file)	Creates a recipe from an existing neural network instance and save it into a .json file.
train_loop(n_epochs[, verbose, validate, ...])	Train the model
transform_batch(x[, y, n_classes])	Transforms a training batch into the format expected by the network.

Attributes

checkpoint_dir
early_stopping_monitor	Sets an early stopping monitor.
log_dir
test_generator
train_generator
val_generator
valid_losses
valid_metrics
valid_optimizers

add_learning_rate_scheduler(scheduler_type='PiecewiseConstantDecay', **kwargs)

Add a learning rate scheduler to the current neural network interface.

Notes: - This method must be called before training and after an optimizer has been defined.

• Keep in mind that in the schedulers a ‘step’ corresponds to each time the optmization algorithm is called. Normally this means that each batch is a step (i.e.: each epoch has several steps).

Args:

scheduler_type:str

One of four scheduler types: PiecewiseConstantDecay , ExponentialDecay, InverseTimeDecay , or PolynomialDecay.

Each type also requires additional arguments, as detailed below.

PiecewiseConstantDecay:

boundaries:list

A list of Tensors or ints or floats with strictly increasing entries, and with all elements having the same type as the optimizer step.

values:list

A list of Tensors or floats or ints that specifies the values for the intervals defined by boundaries. It should have one more element than boundaries, and all elements should have the same type.

ExponentialDecay:

initial_learning_rate:float

The initial learning rate.

decay_steps: int

The decay steps (must be positive). In each step, the learning rate is calculated as initial_learning_rate * decay_rate ^ (step / decay_steps)

decay_rate:float

The decay rate.

staircase:bool

If True decay the learning rate at discrete intervals

InverseTimeDecay:

initial_learning_rate:float

The initial learning rate.

decay_steps:int

How often to apply decay.

decay_rate:float

The decay rate.

staircase:bool

Whether to apply decay in a discrete staircase, as opposed to continuous, fashion.

PolynomialDecay:

initial_learning_rate:float

The initial learning rate.

decay_steps:int

Must be positive. The number of steps in which the end_learning_rate should be reached.

end_learning_rate:float

The minimal end learning rate.

power:float

The power of the polynomial. Defaults to linear, 1.0.

cycle:bool

Whether or not it should cycle beyond decay_steps.

classmethod build(recipe_file)

Build a model from a recipe file

Args:

recipe:str

path to .json file containing the recipe

Returns:

instance:

An instance of the neural network interface

property checkpoint_dir

property early_stopping_monitor

Sets an early stopping monitor.

An early stopping monitor is a dictionary specifying how a target metric should be monitored during training. When the conditions are met, the training loop will be stopped and the model will keep the set of weights that resulted in the best value for the target metric.

The following parameters are expected:

metric: str

The name of the metric to be monitored. It must be one the metrics defined when creating a neural network interface, either through the ‘metrics’ argument of the class constructor or the ‘metrics’ field in a recipe. The name must be prefixed by ‘train_’ or ‘val_’, indicating weather the training or validation metric should be monitored.

decreasing: bool,

If True, improvements will be indicated by a decrease in the metric value during training. If False, improvements will be defined as an increase in the metric value.

period: int

The number of epochs the training loop will continue without any improvement before training is stopped. Example: If period is 5, training will stop if the target metric does not improve for 5 consecutive epochs.

min_epochs: int

The number of epochs to train for before starting to monitor.

delta: float

The minimum difference between the current metric value and the best value recorded since the monitor started. An improvement is only considered if (current value - best value) <= delta (if decreasing is True) or (current value - best value) >= delta (if decreasing is False)

baseline: float or None

If this value is reached, training will stop immediately. If None, this parameter is ignored.

classmethod load(model_file, new_model_folder='kt-tmp', overwrite=True, load_audio_repr=False, replace_top=False, diff_n_classes=None)

Load a model from a ketos (.kt) model file.

Args:

model_file:str

Path to the ketos (.kt) file

new_model_folder:str

Path to folder where files associated with the model will be stored. By default the files will be saved to a folder named ‘kt-tmp’ created in the current working directory.

overwrite: bool

If True, the ‘new_model_folder’ will be overwritten. Default is True.

replace_top: bool

If True, the classification top of the model will be replaced by a new, untrained one. What is actually replaced (i.e.: what exactly is the “top”) is defined by the architecture. It is usually a block of dense layers with the appropriate activations. Default is False.

diff_n_classes: int

Only relevant when ‘replace_top’ is True. If the new model should have a different number of classes it can be specified by this parameter. If left as None, the new model will have the same number of classes as the original model.

load_audio_repr: bool

Depracated. Use ketos.audio.load_audio_representation_from_file instead. If True, also return a dictionary with the audio representation.

Raises:

FileExistsError: If the ‘new_model_folder’ already exists and ‘overwrite’ is False.

Returns:

model_instance:

The loaded model

audio_repr:

Depracated. Use ketos.audio.load_audio_representation_from_file instead. If load_audio_repr is True, also return a dictionary with the audio representation.

property log_dir

run_on_batch(input_batch, return_raw_output=False, transform_input=True)

Run the model on a batch of inputs

Args:

input_batch: numpy.array

The batch of inputs

transform_input:bool

If True, the input_batch is transformed by the interface’s transform_input() method

return_raw_output:bool

If False, the model output will be transformed by transform_output(). If true, the model output will return the output given by the tensorflow predict_on_batch function. In this case, the return is a numpy array(s) of prediction scores with shape (batch_size, n_classes).

Returns:

output

The corresponding batch of model outputs

run_on_instance(input, return_raw_output=False, transform_input=True)

Run the model on one input

Args:

input: numpy.array

The input in the shape expected by transform_input()

return_raw_output:bool

If False, the model output will be transformed by transform_output(). If true, the model output will be returned without any modifications.

transform_input:bool

If True, the input is transformed by the interface’s transform_input() method

Returns:

output

The model output

run_on_test_generator(return_raw_output=False, compute_val_metrics=True, verbose=True, return_metrics=False)

Run the model on the test generator

Args:

return_raw_output:bool

If False, the model output will be transformed by transform_output(). If True, the model output will be returned without any modifications.

compute_val_metrics: bool

If True, compute the same metrics used for validation when running the model on the test generator

verbose: bool

If True and compute_val_metrics is also true, print the results.

return_metrics: bool

Only relevant if ‘compute_val_metrics=True’. If True, return a dictionary with the metrics

Returns:

output

The corresponding batch of model outputs

metrics

A dictionary with the validation metrics. Only returned if both ‘compute_val_metrics’ and ‘return_metrics’ are True.

save(output_name=None, checkpoint_name=None, **kwargs)

Save the trained model to a file.

The model can be saved to three different formats:

• ketos (.kt)

• protobuf (.pb)

• ketos-protobuf (.ktpb), compatible with PAMGuard

The format will be inferred from the extension given to the output file, as follows

• .kt (or any other extension): ketos

• .pb: protobuf

• .ktpb: ketos-protobuf (PAMGuard compatible)

The save method accepts different optional arguments depending on the format. See the documentation of export functions for more information:

• ketos: export.export_to_ketos()

• protobuf: export.export_to_protobuf()

• ketos-protobuf: export.export_to_ketos_protobuf()

Args:

output_name: str

Path to the output file

checkpoint_name: str

The name of the checkpoint to be saved (e.g.:cp-0015.ckpt). If None, will use the latest checkpoints

audio_repr: dict

Audio representation dictionary. Optional for the ketos format (.kt), required for the ketos-protobuf (.ktpb) format, not applicable for the protobuf (.pb) format. It is also possible to specify the path to a json file containing the audio representation.

save_recipe_file(recipe_file)

Creates a recipe from an existing neural network instance and save it into a .json file.

This method is a convenience method that wraps _extract_recipe_dict() and _write_recipe_file()

Args:

recipe_file:str

Path to .json file in which the recipe will be saved.

property test_generator

property train_generator

train_loop(n_epochs, verbose=True, validate=True, log_tensorboard=False, tensorboard_metrics_name='tensorboard_metrics', log_csv=False, csv_name='log.csv', checkpoint_freq=5, early_stopping=False)

Train the model

Typically, before starting the training loop, a few steps will already have been taken:

>>> #Set the batch generator for the training data
>>> model.train_generator = my_train_generator 
>>> #Set the batch generator for the validation data (optional; only if the validate option is set to True)
>>> model.val_generator = my_val_generator 
>>> # Set the log_dir
>>> model.log_dir = "./my_logs" 
>>> # Set the checkpoint_dir
>>> model.checkpoint_dir = "./my_checkpoints" 
>>> model.train_loop(n_epochs=50) 

Args:

n_epochs:int

The number of epochs (i.e.: passes through the entire training dataset, as defined by the train_generator attribute)

verbose:bool

If True, print summary metrics at the end of each epoch

validate:bool

If True, evaluate the model on the validation data (as defined by the val_generator attribute) at the end of each epoch

log_tensorboard:bool

If True, log the training and validation (if validate is True) metrics in the tensoraboard format. See ‘tensorboard_metrics_name’ below.

tensorboard_metrics_name:string

The name of the directory where the tensorboard metrics will be saved. This directory will be created within the path specified by the log_dir attribute. Default is ‘tensorboard_metrics’. Only relevant if log_tensorboard is True.

log_csv:bool

If True, log the training and validation (if validate is True) metrics in a csv file (see csv_name).

The csv will have the following columns,

• epoch: the epoch number, starting from 1

• loss: the value of the loss metric

• dataset: ‘train’ or ‘val’ (only when validate is True)

In addition, each metric defined by the metrics attribute will be added as a column.

Example:

epoch,loss,dataset,CategoricalAccuracy,Precision,Recall 1,0.353,train,0.668,0.653,0.796 1,0.560,val,0.448,0.448,1.0 … 50,0.053,train,0.968,0.953,0.986 50,0.160,val,0.848,0.748,0.838

checkpoint_freq:int

The frequency (in epochs) with which checkpoints (i.e.: the model weights) will be saved to the directory defined by the checkpoint_dir attribute. This number should not exceed ‘n_epochs’. If early_stopping (see below) is used, this parameter is ignored and every epoch is saved as a checkpoint so that the model state can be set to whichever chekpopint is selected by the early stopping monitor. If this value is <=0, no checkpoints will be saved.

early_stopping: bool

If False, train for n_epochs. If True, use the early_stop_monitor to stop training when the conditions defined there are reached (or n_epochs is reached, whichever happens first). When training is stopped by the early stopping monitor, an attribute ‘last_epoch_with_improvement’ will be added to the object. This attribute holds the epoch number (starting from zero) that had the best metric value based on the conditions set by the early_stopping_monitor. The ‘last_epoch_with_improvement’ reflects the current state of the weights when trained is stopped early.

classmethod transform_batch(x, y=None, n_classes=2)

Transforms a training batch into the format expected by the network.

When this interface is subclassed to make new neural_network classes, this method can be overwritten to accomodate any transformations required. Common operations are reshaping of input arrays and parsing or one hot encoding of the labels.

The transform function must accept a batch of inputs and optionally a batch of labels as arguments. The function must return the transformed batch of inputs if given only the inputs without the labels and the transformed batch of input plus the transformed labels if given inputs and labels.

Args:

x:numpy.array

The batch of inputs with shape (batch_size, width, height)

y:numpy.array

The batch of labels. Each label must be represented as an integer, ranging from zero to n_classes The array is expected to have a field named ‘label’.

n_classes:int

The number of possible classes for one hot encoding.

Returns:

X:numpy.array

The transformed batch of inputs

Y:numpy.array

The transformed batch of labels

Examples:

>>> import numpy as np
>>> # Create a batch of 10 5x5 arrays
>>> inputs = np.random.rand(10,5,5)
>>> inputs.shape
(10, 5, 5)

>>> # Create a batch of 10 labels (0 or 1)
>>> labels = np.random.choice([0,1], size=10)
>>> labels.shape
(10,)

>>> transformed_inputs, transformed_labels = NNInterface.transform_batch(inputs, labels, n_classes=2)
>>> transformed_inputs.shape
(10, 5, 5, 1)

>>> transformed_labels.shape
(10, 2)

property val_generator

valid_losses = {'BinaryCrossentropy': <class 'keras.losses.BinaryCrossentropy'>, 'CategoricalCrossentropy': <class 'keras.losses.CategoricalCrossentropy'>, 'CategoricalHinge': <class 'keras.losses.CategoricalHinge'>, 'CosineSimilarity': <class 'keras.losses.CosineSimilarity'>, 'FScoreLoss': <class 'ketos.neural_networks.dev_utils.losses.FScoreLoss'>, 'Hinge': <class 'keras.losses.Hinge'>, 'Huber': <class 'keras.losses.Huber'>, 'KLD': <function kl_divergence>, 'LogCosh': <class 'keras.losses.LogCosh'>, 'MAE': <function mean_absolute_error>, 'MAPE': <function mean_absolute_percentage_error>, 'MSE': <function mean_squared_error>, 'MSLE': <function mean_squared_logarithmic_error>, 'MeanAbsoluteError': <class 'keras.losses.MeanAbsoluteError'>, 'MeanAbsolutePercentageError': <class 'keras.losses.MeanAbsolutePercentageError'>, 'MeanSquaredError': <class 'keras.losses.MeanSquaredError'>, 'MeanSquaredLogarithmicError': <class 'keras.losses.MeanSquaredLogarithmicError'>, 'Poisson': <class 'keras.losses.Poisson'>, 'SparseCategoricalCrossentropy': <class 'keras.losses.SparseCategoricalCrossentropy'>}

valid_metrics = {'AUC': <class 'keras.metrics.AUC'>, 'Accuracy': <class 'keras.metrics.Accuracy'>, 'BinaryAccuracy': <class 'keras.metrics.BinaryAccuracy'>, 'BinaryCrossentropy': <class 'keras.metrics.BinaryCrossentropy'>, 'CategoricalAccuracy': <class 'keras.metrics.CategoricalAccuracy'>, 'CategoricalCrossentropy': <class 'keras.metrics.CategoricalCrossentropy'>, 'CategoricalHinge': <class 'keras.metrics.CategoricalHinge'>, 'CosineSimilarity': <class 'keras.metrics.CosineSimilarity'>, 'FalseNegatives': <class 'keras.metrics.FalseNegatives'>, 'FalsePositives': <class 'keras.metrics.FalsePositives'>, 'Hinge': <class 'keras.metrics.Hinge'>, 'KLDivergence': <class 'keras.metrics.KLDivergence'>, 'LogCoshError': <class 'keras.metrics.LogCoshError'>, 'Mean': <class 'keras.metrics.Mean'>, 'MeanAbsoluteError': <class 'keras.metrics.MeanAbsoluteError'>, 'MeanAbsolutePercentageError': <class 'keras.metrics.MeanAbsolutePercentageError'>, 'MeanIoU': <class 'keras.metrics.MeanIoU'>, 'MeanRelativeError': <class 'keras.metrics.MeanRelativeError'>, 'MeanSquaredError': <class 'keras.metrics.MeanSquaredError'>, 'MeanSquaredLogarithmicError': <class 'keras.metrics.MeanSquaredLogarithmicError'>, 'Poisson': <class 'keras.metrics.Poisson'>, 'Precision': <class 'keras.metrics.Precision'>, 'Recall': <class 'keras.metrics.Recall'>, 'RootMeanSquaredError': <class 'keras.metrics.RootMeanSquaredError'>, 'SensitivityAtSpecificity': <class 'keras.metrics.SensitivityAtSpecificity'>, 'SparseCategoricalAccuracy': <class 'keras.metrics.SparseCategoricalAccuracy'>, 'SparseCategoricalCrossentropy': <class 'keras.metrics.SparseCategoricalCrossentropy'>, 'SparseTopKCategoricalAccuracy': <class 'keras.metrics.SparseTopKCategoricalAccuracy'>, 'SpecificityAtSensitivity': <class 'keras.metrics.SensitivityAtSpecificity'>, 'SquaredHinge': <class 'keras.metrics.SquaredHinge'>, 'Sum': <class 'keras.metrics.Sum'>, 'TopKCategoricalAccuracy': <class 'keras.metrics.TopKCategoricalAccuracy'>, 'TrueNegatives': <class 'keras.metrics.TrueNegatives'>, 'TruePositives': <class 'keras.metrics.TruePositives'>}

valid_optimizers = {'Adadelta': <class 'keras.optimizer_v2.adadelta.Adadelta'>, 'Adagrad': <class 'keras.optimizer_v2.adagrad.Adagrad'>, 'Adam': <class 'keras.optimizer_v2.adam.Adam'>, 'Adamax': <class 'keras.optimizer_v2.adamax.Adamax'>, 'Nadam': <class 'keras.optimizer_v2.nadam.Nadam'>, 'RMSprop': <class 'keras.optimizer_v2.rmsprop.RMSprop'>, 'SGD': <class 'keras.optimizer_v2.gradient_descent.SGD'>}