CNN

cnn sub-module within the ketos.neural_networks module

This module provides classes to implement Convolutional Neural Networks (CNNs).

Contents:

CNN class: CNNInterface class:

class ketos.neural_networks.cnn.CNN1DArch(dense_layers, n_classes, pre_trained_base=None, convolutional_layers=None, **kwargs)[source]

Bases: tensorflow.python.keras.engine.training.Model

Implement an 1D (temporal) Convolutional Neural Network

Note: in addition to the dense layers specified in the ‘dense_layers’ argument, an extra dense

layer will always be added to the end. The output of this layer is determined by the ‘n_classes’ parameter.

Args:
convolutional_layers: list

A list of dictionaries containing the detailed specification for the convolutional layers. Each layer is specified as a dictionary with the following format:

>>> {'n_filters':96, "filter_shape":(11,11), 'strides':4, 'padding':'valid', activation':'relu', 'max_pool': {'pool_size':(3,3) , 'strides':(2,2)}, 'batch_normalization':True} 
dense_layers: list

A list of dictionaries containing the detailed specification for the fully connected layers. Each layer is specified as a dictionary with the following format:

>>> {'n_hidden':4096, 'activation':'relu', 'batch_normalization':True, 'dropout':0.5} 

This list should not include the output layr, which will be automatically added based on the ‘n_classes’ parameter.

n_classes:int

The number of classes the network will be used to classify. The output will be this number of values representing the scores for each class. Scores sum to 1.0.

pre_trained_base: instance of CNN1DArch

A pre-trained CNN 1D model from which the residual blocks will be taken. Use by the the clone_with_new_top method when creating a clone for transfer learning

call(inputs, training=None)[source]

Calls the model on new inputs.

In this case call just reapplies all ops in the graph to the new inputs (e.g. build a new computational graph from the provided inputs).

Arguments:

inputs: A tensor or list of tensors. training: Boolean or boolean scalar tensor, indicating whether to run

the Network in training mode or inference mode.

mask: A mask or list of masks. A mask can be

either a tensor or None (no mask).

Returns:

A tensor if there is a single output, or a list of tensors if there are more than one outputs.

clone_with_new_top(n_classes=None, freeze_base=True)[source]

Clone this instance but replace the original classification top with a new (untrained) one

Args:
n_classes:int

The number of classes the new classification top should output. If None(default), the original number of classes will be used.

freeze_base:bool

If True, the weights of the feature extraction base will be froze (untrainable) in the new model.

Returns:
cloned_model: instance of CNN1DArch

The new model with the old feature extraction base and new classification top.

freeze_conv_block()[source]

Freeze the convolutional block

freeze_top()[source]

Freeze the classification (dense) block

get_feature_extraction_base()[source]

Retrive the feature extraction base (initial convolutional layer + residual blocks)

Returns:

list containing the feature extraction layers

unfreeze_conv_block()[source]

Unfreeze the convolutional block

unfreeze_top()[source]

Unfreeze the classification (dense) block

class ketos.neural_networks.cnn.CNN1DInterface(convolutional_layers=[{'n_filters': 8, 'filter_shape': 128, 'strides': 2, 'padding': 'causal', 'activation': 'relu', 'max_pool': None, 'batch_normalization': True}, {'n_filters': 16, 'filter_shape': 64, 'strides': 2, 'padding': 'causal', 'activation': 'relu', 'max_pool': {'pool_size': 8, 'strides': 8}, 'batch_normalization': True}, {'n_filters': 32, 'filter_shape': 32, 'strides': 2, 'padding': 'causal', 'activation': 'relu', 'max_pool': {'pool_size': 8, 'strides': 8}, 'batch_normalization': True}, {'n_filters': 64, 'filter_shape': 16, 'strides': 2, 'padding': 'causal', 'activation': 'relu', 'max_pool': None, 'batch_normalization': True}, {'n_filters': 128, 'filter_shape': 8, 'strides': 2, 'padding': 'causal', 'activation': 'relu', 'max_pool': None, 'batch_normalization': True}, {'n_filters': 256, 'filter_shape': 4, 'strides': 2, 'padding': 'causal', 'activation': 'relu', 'max_pool': {'pool_size': 4, 'strides': 4}, 'batch_normalization': True}], dense_layers=[{'n_hidden': 512, 'activation': 'relu', 'batch_normalization': True, 'dropout': 0.5}, {'n_hidden': 128, 'activation': 'relu', 'batch_normalization': True, 'dropout': 0.5}], n_classes=2, optimizer=Adam ketos recipe, loss_function=CategoricalCrossentropy ketos recipe, metrics=[CategoricalAccuracy ketos recipe, Precision ketos recipe, Recall ketos recipe])[source]

Bases: ketos.neural_networks.cnn.CNNInterface

Create an 1D (temporal) CNN model with the standardized Ketos interface.

Args:
convolutional_layers: list

A list of dictionaries containing the detailed specification for the convolutional layers. Each layer is specified as a dictionary with the following format:

>>> {'n_filters':96, "filter_shape":(11,11), 'strides':4, 'padding':'valid', activation':'relu', 'max_pool': {'pool_size':(3,3) , 'strides':(2,2)}, 'batch_normalization':True} 
dense_layers: list

A list of dictionaries containing the detailed specification for the fully connected layers. Each layer is specified as a dictionary with the following format:

>>> {'n_hidden':4096, 'activation':'relu', 'batch_normalization':True, 'dropout':0.5} 
n_classes:int

The number of classes the network will be used to classify. The output will be this number of values representing the scores for each class. Scores sum to 1.0.

optimizer: ketos.neural_networks.RecipeCompat object

A recipe compatible optimizer (i.e.: wrapped by the ketos.neural_networksRecipeCompat class)

loss_function: ketos.neural_networks.RecipeCompat object

A recipe compatible loss_function (i.e.: wrapped by the ketos.neural_networksRecipeCompat class)

metrics: list of ketos.neural_networks.RecipeCompat objects

A list of recipe compatible metrics (i.e.: wrapped by the ketos.neural_networksRecipeCompat class). These metrics will be computed on each batch during training.

Examples:

>>> # Most users will create a model based on a Ketos recipe 
>>> # The one below, specifies a CNN with 3 convolutional layers and 2 dense layers
>>>
>>> recipe = {'conv_set':[[64, False], [128, True], [256, True]], 
...   'dense_set': [512, ],
...   'n_classes':2,
...   'optimizer': {'name':'Adam', 'parameters': {'learning_rate':0.005}},
...   'loss_function': {'name':'FScoreLoss', 'parameters':{}},  
...   'metrics': [{'name':'CategoricalAccuracy', 'parameters':{}}]
... }
>>> # To create the CNN, simply  use the  'build_from_recipe' method:
>>> cnn = CNNInterface.build_from_recipe(recipe, recipe_compat=False) 
classmethod transform_batch(x, y, y_fields=['label'], n_classes=2)[source]

Transforms a training batch into the format expected by the network.

When this interface is subclassed to make new neural_network classes, this method can be overwritten to accomodate any transformations required. Common operations are reshaping of input arrays and parsing or one hot encoding of the labels.

Args:
x:numpy.array

The batch of inputs with shape (batch_size, width, height)

y:numpy.array

The batch of labels. Each label must be represented as an integer, ranging from zero to n_classes The array is expected to have a field named ‘label’.

n_classes:int

The number of possible classes for one hot encoding.

Returns:
X:numpy.array

The transformed batch of inputs

Y:numpy.array

The transformed batch of labels

Examples:

>>> import numpy as np
>>> # Create a batch of 10 5x5 arrays
>>> inputs = np.random.rand(10,5,5)
>>> inputs.shape
(10, 5, 5)
>>> # Create a batch of 10 labels (0 or 1)
>>> labels = np.random.choice([0,1], size=10).astype([('label','<i4')])
>>> labels.shape
(10,)
>>> transformed_inputs, transformed_labels = NNInterface.transform_batch(inputs, labels, n_classes=2)
>>> transformed_inputs.shape
(10, 5, 5, 1)
>>> transformed_labels.shape
(10, 2)
class ketos.neural_networks.cnn.CNNArch(convolutional_layers, dense_layers, n_classes, pre_trained_base=None, **kwargs)[source]

Bases: tensorflow.python.keras.engine.training.Model

Implement a Convolutional Neural Network

Note: in addition to the dense layers specified in the ‘dense_layers’ argument, an extra dense

layer will always be added to the end. The output of this layer is determined by the ‘n_classes’ parameter.

Args:
convolutional_layers: list

A list of dictionaries containing the detailed specification for the convolutional layers. Each layer is specified as a dictionary with the following format:

>>> {'n_filters':96, "filter_shape":(11,11), 'strides':4, 'padding':'valid', activation':'relu', 'max_pool': {'pool_size':(3,3) , 'strides':(2,2)}, 'batch_normalization':True} 
dense_layers: list

A list of dictionaries containing the detailed specification for the fully connected layers. Each layer is specified as a dictionary with the following format:

>>> {'n_hidden':4096, 'activation':'relu', 'batch_normalization':True, 'dropout':0.5} 

This list should not include the output layr, which will be automatically added based on the ‘n_classes’ parameter.

pre_trained_base: instance of CNNArch

A pre-trained CNN model from which the residual blocks will be taken. Use by the the clone_with_new_top method when creating a clone for transfer learning

n_classes:int

The number of classes the network will be used to classify. The output will be this number of values representing the scores for each class. Scores sum to 1.0.

call(inputs, training=None)[source]

Calls the model on new inputs.

In this case call just reapplies all ops in the graph to the new inputs (e.g. build a new computational graph from the provided inputs).

Arguments:

inputs: A tensor or list of tensors. training: Boolean or boolean scalar tensor, indicating whether to run

the Network in training mode or inference mode.

mask: A mask or list of masks. A mask can be

either a tensor or None (no mask).

Returns:

A tensor if there is a single output, or a list of tensors if there are more than one outputs.

clone_with_new_top(n_classes=None, freeze_base=True)[source]

Clone this instance but replace the original classification top with a new (untrained) one

Args:
n_classes:int

The number of classes the new classification top should output. If None(default), the original number of classes will be used.

freeze_base:bool

If True, the weights of the feature extraction base will be froze (untrainable) in the new model.

Returns:
cloned_model: instance of CNNArch

The new model with the old feature extraction base and new classification top.

freeze_conv_block()[source]

Freeze the convolutional block

freeze_top()[source]

Freeze the classification (dense) block

get_feature_extraction_base()[source]

Retrive the feature extraction base (initial convolutional layer + residual blocks)

Returns:

list containing the feature extraction layers

unfreeze_conv_block()[source]

Unfreeze the convolutional block

unfreeze_top()[source]

Unfreeze the classification (dense) block

class ketos.neural_networks.cnn.CNNInterface(convolutional_layers=[{'n_filters': 32, 'filter_shape': (8, 8), 'strides': 4, 'padding': 'valid', 'activation': 'relu', 'max_pool': {'pool_size': (3, 3), 'strides': (2, 2)}, 'batch_normalization': True}, {'n_filters': 64, 'filter_shape': (3, 3), 'strides': 1, 'padding': 'valid', 'activation': 'relu', 'max_pool': {'pool_size': (3, 3), 'strides': (2, 2)}, 'batch_normalization': True}], dense_layers=[{'n_hidden': 512, 'activation': 'relu', 'batch_normalization': True, 'dropout': 0.5}, {'n_hidden': 128, 'activation': 'relu', 'batch_normalization': True, 'dropout': 0.5}], n_classes=2, optimizer=Adam ketos recipe, loss_function=BinaryCrossentropy ketos recipe, metrics=[BinaryAccuracy ketos recipe])[source]

Bases: ketos.neural_networks.dev_utils.nn_interface.NNInterface

Creates a CNN model with the standardized Ketos interface.

Args:
convolutional_layers: list

A list of dictionaries containing the detailed specification for the convolutional layers. Each layer is specified as a dictionary with the following format:

>>> {'n_filters':96, "filter_shape":(11,11), 'strides':4, 'padding':'valid', activation':'relu', 'max_pool': {'pool_size':(3,3) , 'strides':(2,2)}, 'batch_normalization':True} 
dense_layers: list

A list of dictionaries containing the detailed specification for the fully connected layers. Each layer is specified as a dictionary with the following format:

>>> {'n_hidden':4096, 'activation':'relu', 'batch_normalization':True, 'dropout':0.5} 
n_classes:int

The number of classes the network will be used to classify. The output will be this number of values representing the scores for each class. Scores sum to 1.0.

optimizer: ketos.neural_networks.RecipeCompat object

A recipe compatible optimizer (i.e.: wrapped by the ketos.neural_networksRecipeCompat class)

loss_function: ketos.neural_networks.RecipeCompat object

A recipe compatible loss_function (i.e.: wrapped by the ketos.neural_networksRecipeCompat class)

metrics: list of ketos.neural_networks.RecipeCompat objects

A list of recipe compatible metrics (i.e.: wrapped by the ketos.neural_networksRecipeCompat class). These metrics will be computed on each batch during training.

Examples:

>>> # Most users will create a model based on a Ketos recipe 
>>> # The one below, specifies a CNN with 3 convolutional layers and 2 dense layers
>>>
>>> recipe = {'conv_set':[[64, False], [128, True], [256, True]], 
...   'dense_set': [512, ],
...   'n_classes':2,
...   'optimizer': {'name':'Adam', 'parameters': {'learning_rate':0.005}},
...   'loss_function': {'name':'FScoreLoss', 'parameters':{}},  
...   'metrics': [{'name':'CategoricalAccuracy', 'parameters':{}}]
... }
>>> # To create the CNN, simply  use the  'build_from_recipe' method:
>>> cnn = CNNInterface.build_from_recipe(recipe, recipe_compat=False)