Neural-Network Interface

class ketos.neural_networks.dev_utils.nn_interface.NNInterface(optimizer, loss_function, metrics)[source]

Bases: object

General interface for neural network architectures in the ketos.neural_networks module.

This class implements common methods for neural network models and is supposed to be subclassed. When implementing new neural network architectures, the interface implemented in this clas can be inherited.


optimizer: RecipeCompat object

An instance of the RecipeCompat class wrapping a tensorflow(-compatible) optimizer (e.g.:from tensorflow.keras.optimizers)

loss_function: RecipeCompat object

An instance of the RecipeCompat class wrappinf a tensorflow(-compatible) loss-function (e.g.:from tensorflow.keras.losses)

metrics: list of RecipeCompat objects

A list of instances of the RecipeCompat class wrapping a tensorflow(-compatible) metric (e.g.:from tensorflow.keras.metrics)


The following example shows how a newly defined network architecture could use the interface provided by NNInterface.

First, the new architecture must be defined. Here, a simple multi-layer perceptron is defined in the following class.

>>> class MLP(tf.keras.Model): 
...    def __init__(self, n_neurons, activation):
...        super(MLP, self).__init__()
...        self.dense = tf.keras.layers.Dense(n_neurons, activation=activation)
...        self.final_node = tf.keras.layers.Dense(1)
...    def call(self, inputs):
...        output = self.dense(inputs)
...        output = self.dense(output)
...        output = self.final_node(output)

With the architecture, the interface to the MLP can be created by subclassing NNInterface:

from ketos.neural_networks.dev_utils import RecipeCompat, NNInterface

>>> class MLPInterface(NNInterface):  
...    @classmethod
...    def _build_from_recipe(cls, recipe, recipe_compat=True):
...        n_neurons = recipe['n_neurons']    # take the n_neurons parameter from the recipe instead of using the default
...        activation = recipe['activation']  # take the activation parameter from the recipe instead of using the default
...         if recipe_compat == True:
...            optimizer = recipe['optimizer']
...            loss_function = recipe['loss_function']
...            metrics = recipe['metrics']
...        else:
...            optimizer = cls._optimizer_from_recipe(recipe['optimizer'])
...            loss_function = cls._loss_function_from_recipe(recipe['loss_function'])
...            metrics = cls._metrics_from_recipe(recipe['metrics'])
...        instance = cls(n_neurons=n_neurons, activation=activation, optimizer=optimizer, loss_function=loss_function, metrics=metrics)
...        return instance
...  @classmethod
...  def _read_recipe_file(cls, json_file, return_recipe_compat=True):
...        with open(json_file, 'r') as json_recipe:
...            recipe_dict = json.load(json_recipe)
...        optimizer = cls.optimizer_from_recipe(recipe_dict['optimizer'])
...        loss_function = cls.loss_function_from_recipe(recipe_dict['loss_function'])
...        metrics = cls.metrics_from_recipe(recipe_dict['metrics'])
...        if return_recipe_compat == True:
...            recipe_dict['optimizer'] = optimizer
...            recipe_dict['loss_function'] = loss_function
...            recipe_dict['metrics'] = metrics
...        else:
...            recipe_dict['optimizer'] = cls._optimizer_to_recipe(optimizer)
...            recipe_dict['loss_function'] = cls._loss_function_to_recipe(loss_function)
...            recipe_dict['metrics'] = cls._metrics_to_recipe(metrics)
...        recipe_dict['n_neurons'] = recipe_dict['n_neurons']    # read the n_neurons parameter from the recipe file
...        recipe_dict['activation'] = recipe_dict['activation']  # read the activation parameter from the recipe file
...        return recipe_dict
...     def __init__(self, n_neurons, activation, optimizer, loss_function, metrics):
...        super(MLPInterface, self).__init__(optimizer, loss_function, metrics)
...        self.n_neurons = n_neurons
...        self.activation = activation
...        self.model = MLP(n_neurons=n_neurons, activation=activation)
...    def _extract_recipe_dict(self):
...        recipe = {}
...        recipe['optimizer'] = self._optimizer_to_recipe(self.optimizer)
...        recipe['loss_function'] = self._loss_function_to_recipe(self.loss_function)
...        recipe['metrics'] = self._metrics_to_recipe(self.metrics)
...        recipe['n_neurons'] = self.n_neurons
...        recipe['activation'] = self.activation
...        return recipe
add_learning_rate_scheduler(scheduler_type='PiecewiseConstantDecay', **kwargs)[source]

Add a learning rate scheduler to the current neural network interface.

Notes: - This method must be called before training and after an optimizer has been defined.
  • Keep in mind that in the schedulers a ‘step’ corresponds to each time the optmization algorithm is called. Normally this means that each batch is a step (i.e.: each epoch has several steps).


One of four scheduler types: ‘PiecewiseConstantDecay’, ‘ExponentialDecay’, ‘InverseTimeDecay’ or ‘PolynomialDecay’.

Each type also requires additional arguments:

‘PiecewiseConstantDecay’ (See the tensorflow documentation)

A list of Tensors or ints or floats with strictly increasing entries, and with all elements having the same type as the optimizer step.


A list of Tensors or floats or ints that specifies the values for the intervals defined by boundaries. It should have one more element than boundaries, and all elements should have the same type.

‘ExponentialDecay’ (See the tensorflow documentation)

The initial learning rate.

decay_steps: int

The decay steps (must be positive). In each step, the learning rate is calculated as initial_learning_rate * decay_rate ^ (step / decay_steps)


The decay rate.


If True decay the learning rate at discrete intervals

‘InverseTimeDecay’ (See the tensorflow documentation)

The initial learning rate.


How often to apply decay.


The decay rate.


Whether to apply decay in a discrete staircase, as opposed to continuous, fashion.

‘PolynomialDecay’ (See the tensorflow documentation)

The initial learning rate.


Must be positive. The number of steps in which the end_learning_rate should be reached.


The minimal end learning rate.


The power of the polynomial. Defaults to linear, 1.0.


Whether or not it should cycle beyond decay_steps.

classmethod build_from_recipe_file(recipe_file)[source]

Build a model from a recipe file


path to .json file containing the recipe


An instance of the neural network interface

property checkpoint_dir
property early_stopping_monitor

Sets an early stopping monitor.

An early stopping monitor is a dictionary specifying how a target metric should be monitored during training. When the conditions are met, the training loop will be stopped and the model will keep the set of weights that resulted in the best value for the target metric.

The following parameters are expected:

“metric”: str

The name of the metric to be monitored. It must be one the metrics defined when creating a neural network interface, either through the ‘metrics’ argument of the class constructor or the ‘metrics’ field in a recipe. The name must be prefixed by ‘train_’ or ‘val_’, indicating weather the training or validation metric should be monitored.

“decreasing”: bool,

If True, improvements will be indicated by a decrease in the metric value during training. If False, improvements will be defined as an increase in the metric value.

“period”: int

The number of epochs the training loop will continue without any improvement before training is stopped. Example: If period is 5, training will stop if the target metric does not improve for 5 consecutive epochs.

“min_epochs”: int

The number of epochs to train for before starting to monitor.


The minimum difference between the current metric value and the best value recorded since the monitor started. An improvement is only considered if

(current value - best value) <= delta (if decreasing is True) or (current value - best value) >= delta (if decreasing is False)

“baseline”:float or None

If this value is reached, training will stop immediately. If None, this parameter is ignored.

classmethod load_model_file(model_file, new_model_folder, overwrite=True, load_audio_repr=False, replace_top=False, diff_n_classes=None)[source]

Load a model from a ketos (.kt) model file.


Path to the ketos(.kt) file


Path to folder where files associated with the model will be stored.

overwrite: bool

If True, the ‘new_model_folder’ will be overwritten.

replace_top: bool

If True, the classification top of the model will be replaced by a new, untrained one. What is actually replaced (i.e.: what exactly is the “top”) is defined by the architecture. It is usually a block of dense layers with the appropriate activations. Default is False.

diff_n_classes: int

Only relevant when ‘replace_top’ is True. If the new model should have a different number of classes it can be specified by this parameter. If left to none, the new model will have the same number of classes as the original.

load_audio_repr: bool

If True, look for an audio representation included with the model.


FileExistsError: If the ‘new_model_folder’ already exists and ‘overwite’ is False.


model_instance: The loaded model audio_repr: If load_audio_repr is True, also return a dictionary with the loaded audio representation.

property log_dir
run_on_batch(input_batch, return_raw_output=False, transform_input=True)[source]

Run the model on a batch of inputs

input_batch: numpy.array

The batch of inputs


If True, the input_batch is transformed by the interface’s transform_input() method


If False, the model output will be transformed by transform_output(). If true, the model output will be returned without any modifications.


The corresponding batch of model outputs

run_on_instance(input, return_raw_output=False, transform_input=True)[source]

Run the model on one input

input: numpy.array

The input in the shape expected by transform_input()


If False, the model output will be transformed by transform_output(). If true, the model output will be returned without any modifications.


If True, the input is transformed by the interface’s transform_input() method


The model output

run_on_test_generator(return_raw_output=False, compute_val_metrics=True, verbose=True)[source]

Run the model on the test generator


If False, the model output will be transformed by transform_output(). If True, the model output will be returned without any modifications.

compute_val_metrics: bool

If True, compute the same metrics used for validation when running the model on the test generator

verbose: bool

If True and compute_val_metrics is also true, print the results.


The corresponding batch of model outputs

save_model(model_file, checkpoint_name=None, audio_repr_file=None)[source]

Save the current neural network instance as a ketos (.kt) model file.

The file includes the recipe necessary to build the network architecture and the parameter weights.

model_file: str

Path to the .kt file.

checkpoint_name: str

The name of the checkpoint to be loaded (e.g.:cp-0015.ckpt). If None, will use the latest checkpoints

audio_repr_file: str

Optional path to an audio representation .json file. If passed, it will be added to the .kt file.


Creates a recipe from an existing neural network instance and save it into a .json file.

This method is a convenience method that wraps write_recipe() and write_recipe_file()


Path to .json file in which the recipe will be saved.

property test_generator
property train_generator
train_loop(n_epochs, verbose=True, validate=True, log_tensorboard=False, tensorboard_metrics_name='tensorboard_metrics', log_csv=False, csv_name='log.csv', checkpoint_freq=5, early_stopping=False)[source]

Train the model

Typically, before starting the training loop, a few steps will already have been taken:

#Set the batch generator for the training data model.train_generator = my_train_generator #Set the batch generator for the validation data (optional; only if the validate option is set to True) model.val_generator = my_val_generator # Set the log_dir model.log_dir = “./my_logs” # Set the checkpoint_dir model.checkpoint_dir = “./my_checkpoints” model.train_loop(n_epochs=50)


The number of epochs (i.e.: passes through the entire training dataset, as defined by the train_generator attribute)


If True, print summary metrics at the end of each epoch


If True, evaluate the model on the validation data (as defined by the val_generator attribute) at the end of each epoch


If True, log the training and validation (if validate is True) metrics in the tensoraboard format. See ‘tensorboard_metrics_name’ below.


The name of the directory where the tensorboard metrics will be saved. This directory will be created within the path specified by the log_dir attribute. Default is ‘tensorboard_metrics’. Only relevant if log_tensorboard is True.


If True, log the training and validation (if validate is True) metrics in a csv file (see csv_name).

The csv will have the following columns: epoch: the epoch number, starting from 1 loss: the value of the loss metric dataset: ‘train’ or ‘val’(only when validate is True) In addition, each metric defined by the metrics attribute will be added as a column.


epoch,loss,dataset,CategoricalAccuracy,Precision,Recall 1,0.353,train,0.668,0.653,0.796 1,0.560,val,0.448,0.448,1.0 … 50,0.053,train,0.968,0.953,0.986 50,0.160,val,0.848,0.748,0.838


The frequency (in epochs) with which checkpoints (i.e.: the model weights) will be saved to the directory defined by the checkpoint_dir attribute.

early_stopping: bool

If False, train for n_epochs. If True, use the early_stop_monitor to stop training when the conditions defined there are reached (or n_epochs is reached, whichever happens first). When training is stopped by the early stopping monitor, an attribute ‘last_epoch_with_improvement’ will be added to the object. This attribute holds the epoch number (starting from zero) that had the best metric value based on the conditions set by the early_stopping_monitor. The ‘last_epoch_with_improvement’ reflects the current state of the weights when trained is stopped early.

classmethod transform_batch(x, y, y_fields=['label'], n_classes=2)[source]

Transforms a training batch into the format expected by the network.

When this interface is subclassed to make new neural_network classes, this method can be overwritten to accomodate any transformations required. Common operations are reshaping of input arrays and parsing or one hot encoding of the labels.


The batch of inputs with shape (batch_size, width, height)


The batch of labels. Each label must be represented as an integer, ranging from zero to n_classes The array is expected to have a field named ‘label’.


The number of possible classes for one hot encoding.


The transformed batch of inputs


The transformed batch of labels

>>> import numpy as np
>>> # Create a batch of 10 5x5 arrays
>>> inputs = np.random.rand(10,5,5)
>>> inputs.shape
(10, 5, 5)
>>> # Create a batch of 10 labels (0 or 1)
>>> labels = np.random.choice([0,1], size=10).astype([('label','<i4')])
>>> labels.shape
>>> transformed_inputs, transformed_labels = NNInterface.transform_batch(inputs, labels, n_classes=2)
>>> transformed_inputs.shape
(10, 5, 5, 1)
>>> transformed_labels.shape
(10, 2)
property val_generator
valid_losses = {'BinaryCrossentropy': <class 'tensorflow.python.keras.losses.BinaryCrossentropy'>, 'CategoricalCrossentropy': <class 'tensorflow.python.keras.losses.CategoricalCrossentropy'>, 'CategoricalHinge': <class 'tensorflow.python.keras.losses.CategoricalHinge'>, 'CosineSimilarity': <class 'tensorflow.python.keras.losses.CosineSimilarity'>, 'FScoreLoss': <class 'ketos.neural_networks.dev_utils.losses.FScoreLoss'>, 'Hinge': <class 'tensorflow.python.keras.losses.Hinge'>, 'Huber': <class 'tensorflow.python.keras.losses.Huber'>, 'KLD': <function kullback_leibler_divergence>, 'LogCosh': <class 'tensorflow.python.keras.losses.LogCosh'>, 'MAE': <function mean_absolute_error>, 'MAPE': <function mean_absolute_percentage_error>, 'MSE': <function mean_squared_error>, 'MSLE': <function mean_squared_logarithmic_error>, 'MeanAbsoluteError': <class 'tensorflow.python.keras.losses.MeanAbsoluteError'>, 'MeanAbsolutePercentageError': <class 'tensorflow.python.keras.losses.MeanAbsolutePercentageError'>, 'MeanSquaredError': <class 'tensorflow.python.keras.losses.MeanSquaredError'>, 'MeanSquaredLogarithmicError': <class 'tensorflow.python.keras.losses.MeanSquaredLogarithmicError'>, 'Poisson': <class 'tensorflow.python.keras.losses.Poisson'>, 'SparseCategoricalCrossentropy': <class 'tensorflow.python.keras.losses.SparseCategoricalCrossentropy'>}
valid_metrics = {'AUC': <class 'tensorflow.python.keras.metrics.AUC'>, 'Accuracy': <class 'tensorflow.python.keras.metrics.Accuracy'>, 'BinaryAccuracy': <class 'tensorflow.python.keras.metrics.BinaryAccuracy'>, 'BinaryCrossentropy': <class 'tensorflow.python.keras.metrics.BinaryCrossentropy'>, 'CategoricalAccuracy': <class 'tensorflow.python.keras.metrics.CategoricalAccuracy'>, 'CategoricalCrossentropy': <class 'tensorflow.python.keras.metrics.CategoricalCrossentropy'>, 'CategoricalHinge': <class 'tensorflow.python.keras.metrics.CategoricalHinge'>, 'CosineSimilarity': <class 'tensorflow.python.keras.metrics.CosineSimilarity'>, 'FalseNegatives': <class 'tensorflow.python.keras.metrics.FalseNegatives'>, 'FalsePositives': <class 'tensorflow.python.keras.metrics.FalsePositives'>, 'Hinge': <class 'tensorflow.python.keras.metrics.Hinge'>, 'KLDivergence': <class 'tensorflow.python.keras.metrics.KLDivergence'>, 'LogCoshError': <class 'tensorflow.python.keras.metrics.LogCoshError'>, 'Mean': <class 'tensorflow.python.keras.metrics.Mean'>, 'MeanAbsoluteError': <class 'tensorflow.python.keras.metrics.MeanAbsoluteError'>, 'MeanAbsolutePercentageError': <class 'tensorflow.python.keras.metrics.MeanAbsolutePercentageError'>, 'MeanIoU': <class 'tensorflow.python.keras.metrics.MeanIoU'>, 'MeanRelativeError': <class 'tensorflow.python.keras.metrics.MeanRelativeError'>, 'MeanSquaredError': <class 'tensorflow.python.keras.metrics.MeanSquaredError'>, 'MeanSquaredLogarithmicError': <class 'tensorflow.python.keras.metrics.MeanSquaredLogarithmicError'>, 'Poisson': <class 'tensorflow.python.keras.metrics.Poisson'>, 'Precision': <class 'tensorflow.python.keras.metrics.Precision'>, 'Recall': <class 'tensorflow.python.keras.metrics.Recall'>, 'RootMeanSquaredError': <class 'tensorflow.python.keras.metrics.RootMeanSquaredError'>, 'SensitivityAtSpecificity': <class 'tensorflow.python.keras.metrics.SensitivityAtSpecificity'>, 'SparseCategoricalAccuracy': <class 'tensorflow.python.keras.metrics.SparseCategoricalAccuracy'>, 'SparseCategoricalCrossentropy': <class 'tensorflow.python.keras.metrics.SparseCategoricalCrossentropy'>, 'SparseTopKCategoricalAccuracy': <class 'tensorflow.python.keras.metrics.SparseTopKCategoricalAccuracy'>, 'SpecificityAtSensitivity': <class 'tensorflow.python.keras.metrics.SensitivityAtSpecificity'>, 'SquaredHinge': <class 'tensorflow.python.keras.metrics.SquaredHinge'>, 'Sum': <class 'tensorflow.python.keras.metrics.Sum'>, 'TopKCategoricalAccuracy': <class 'tensorflow.python.keras.metrics.TopKCategoricalAccuracy'>, 'TrueNegatives': <class 'tensorflow.python.keras.metrics.TrueNegatives'>, 'TruePositives': <class 'tensorflow.python.keras.metrics.TruePositives'>}
valid_optimizers = {'Adadelta': <class 'tensorflow.python.keras.optimizer_v2.adadelta.Adadelta'>, 'Adagrad': <class 'tensorflow.python.keras.optimizer_v2.adagrad.Adagrad'>, 'Adam': <class 'tensorflow.python.keras.optimizer_v2.adam.Adam'>, 'Adamax': <class 'tensorflow.python.keras.optimizer_v2.adamax.Adamax'>, 'Nadam': <class 'tensorflow.python.keras.optimizer_v2.nadam.Nadam'>, 'RMSprop': <class 'tensorflow.python.keras.optimizer_v2.rmsprop.RMSprop'>, 'SGD': <class 'tensorflow.python.keras.optimizer_v2.gradient_descent.SGD'>}
class ketos.neural_networks.dev_utils.nn_interface.RecipeCompat(recipe_name, template, **kwargs)[source]

Bases: object

Makes a loss function, metric or optimizer compatible with the Ketos recipe format.

The resulting object can be included in a ketos recipe and read by the NNInterface (or it’s subclasses)

recipe_name: str

The name to be used in the recipe

template: constructor

The loss function, metric or optimizer constructor


Any keyword arguments to be passed to the constructor (func)


A RecipeCompat object

>>> # Example Metric
>>> p = tf.keras.metrics.Precision
>>> dec_p = RecipeCompat("precision", p)
>>> # Example Optimizer
>>> opt = tf.keras.optimizers.Adam
>>> dec_opt = RecipeCompat("adam", opt, learning_rate=0.001)
>>> # Example Loss
>>> loss = tf.keras.losses.BinaryCrossentropy
>>> dec_loss = RecipeCompat('binary_crossentropy', loss, from_logits=True)