xfer.BnnRepurposer¶
-
class
xfer.
BnnRepurposer
(source_model: mxnet.module.module.Module, feature_layer_names, context_function=<function cpu>, num_devices=1, bnn_context_function=<function cpu>, sigma=100.0, num_layers=1, n_hidden=10, num_samples_mc=3, learning_rate=0.001, batch_size=20, num_epochs=200, start_annealing=None, end_annealing=None, num_samples_mc_prediction=100, verbose=0)¶ Bases:
xfer.meta_model_repurposer.MetaModelRepurposer
Perform Transfer Learning through a Bayesian Neural Network (BNN) meta-model which repurposes the source neural network.
Parameters: - source_model (
mxnet.mod.Module
) – Source neural network to do transfer learning from. - feature_layer_names (list[str]) – Name of layer(s) in source_model from which features should be transferred.
- context_function (function(int)->:class:mx.context.Context) – MXNet context function that provides device type context. It is used to extract features with the source model.
- num_devices (int) – Number of devices to use to extract features from source_model.
- bnn_context_function (function(int)->:class:mx.context.Context) – MXNet context function used to train the BNN.
- sigma (float) – Standard deviation of the Gaussian prior used for the weights of the BNN meta-model (w_i sim N(0, sigma^2)).
- num_layers (int) – Number of layers of the BNN meta-model.
- n_hidden (int) – Dimensionality of the hidden layers of the BNN meta-model (all hidden layers have the same dimensionality).
- num_samples_mc (int) – Number of samples used for the Monte Carlo approximation of the variational bound.
- learning_rate (float) – Learning rate for the BNN meta-model training.
- batch_size (int) – Mini-batch size for the BNN meta-model training.
- num_epochs (int) – Number of epochs for the BNN meta-model training.
- start_annealing (int) – To help the training of the BNN meta-model, we anneal the KL term using a weight that varies fromm zero to one. start_annealing determines the epoch at which the annealing weight start to increase linearly until it reaches one in the epoch given by end_annealing.
- end_annealing (int) – Determines the epoch at which the annealing process of the KL term ends.
- step_annealing_sample_weight (float) – Amount that the annealing weight is incremented in each epoch (from start_annealing to end_annealing).
- num_samples_mc_prediction (int) – Number of Monte Carlo samples to use on prediction.
- verbose (bool) – Flag to control whether accuracy monitoring is logged during repurposing.
- annealing_weight (float) – Annealing weight in the current epoch.
- train_acc (list[float]) – Accuracy in training set in each epoch.
- test_acc (list[float]) – Accuracy in validation set in each epoch.
- moving_loss_total (list[float]) – Total loss (negative ELBO) smoothed across epochs.
- average_loss (list[float]) – Average loss (negative ELBO) per data point.
- anneal_weights (list[float]) – Annealing weight used in each epoch.
Methods
__init__
Initialize self. deserialize
Uses dictionary to set attributes of repurposer. get_features_from_source_model
Extract feature outputs from feature_layer_names in source_model, merge and return all features and labels. get_params
Get parameters of repurposer that are in the constructor. predict_label
Predict class labels on test data using the target_model (repurposed meta-model). predict_probability
Predict class probabilities on test data using the target_model (repurposed meta-model). repurpose
Train a meta-model using features extracted from training data through the source neural network. save_repurposer
Serialize the repurposed model (source_model, target_model and supporting info) and save it to given file_path. serialize
Saves repurposer (excluding source model) to file_prefix.json, file_prefix_posterior.json, file_prefix_posterior_params.npz. Attributes
end_annealing
Determines the epoch at which the annealing process of the KL term ends. feature_layer_names
Names of the layers to extract features from. source_model
Model to extract features from. start_annealing
Determines the epoch at which the annealing process of the KL term starts. -
start_annealing
¶ Determines the epoch at which the annealing process of the KL term starts.
-
end_annealing
¶ Determines the epoch at which the annealing process of the KL term ends.
-
feature_layer_names
¶ Names of the layers to extract features from.
-
get_features_from_source_model
(data_iterator: mxnet.io.io.DataIter)¶ Extract feature outputs from feature_layer_names in source_model, merge and return all features and labels.
In addition, return mapping of feature_layer_name to indices in feature array.
Parameters: data_iterator ( mxnet.io.DataIter
) – Iterator for data to be passed through the source network and extract features.Returns: features, feature_indices_per_layer and labels. Return type: MetaModelData
-
predict_label
(test_iterator: mxnet.io.io.DataIter)¶ Predict class labels on test data using the target_model (repurposed meta-model).
Parameters: test_iterator (mxnet.io.DataIter) – Test data iterator to return predictions for. Returns: Predicted labels. Return type: numpy.ndarray
-
predict_probability
(test_iterator: mxnet.io.io.DataIter)¶ Predict class probabilities on test data using the target_model (repurposed meta-model).
Parameters: test_iterator (mxnet.io.DataIter) – Test data iterator to return predictions for. Returns: Predicted probabilities. Return type: numpy.ndarray
-
repurpose
(train_iterator: mxnet.io.io.DataIter)¶ Train a meta-model using features extracted from training data through the source neural network.
Set self.target_model to the trained meta-model.
Parameters: train_iterator – Training data iterator to use to extract features from source_model.
-
save_repurposer
(model_name, model_directory='', save_source_model=None)¶ Serialize the repurposed model (source_model, target_model and supporting info) and save it to given file_path.
Parameters: - model_name (str) – Name to save repurposer to.
- model_directory (str) – File directory to save repurposer in.
- save_source_model (boolean) – Flag to choose whether to save repurposer source model. Will use default if set to None. (MetaModelRepurposer default: True, NeuralNetworkRepurposer default: False)
-
source_model
¶ Model to extract features from.
- source_model (