Скачать или смотреть TF SavedModel vs HDF5: How to save TF & Keras Models with Custom Objects?

For all tutorials: muratkarakaya.net
Google Colab: https://colab.research.google.com/dri...
Github pages: https://kmkarakaya.github.io/Deep-Lea...
Medium:   / save-load-keras-models-with-custom-layers  
Controllable Text Generation with Transformers:    • Controllable Text Generation with Transfor...  

How to save TF & Keras Models with Custom Objects?
In this tutorial, we will design and train a Keras model (miniature GPT3) with some custom objects (custom layers). Our aim is to learn how to save the trained model as a whole and load the saved model. We will learn 2 different save formats: "TensorFlow SevedModel" & "HDF5".
References
Let me ask you: Have you ever used a custom layer / loss function / activation function / metric?
What to save
What do we save when we save the whole model?
Why do we save the whole TensorFlow & Keras Models?
What are the Save Formats in TensorFlow / Keras?
Which save format should I use?
Why should I use the TensorFlow SavedModel format?
Which methods / functions to use to save the entire TensorFlow / Keras Models?
How to instruct the method to save in which format?
Create a Keras Model with custom objects ( custom layers)
First, prepare the random train dataset:
Then, build a TensorFlow Data Pipeline
Design a Miniature GPT3 Model with custom layers
We define the first custom layer
the second custom layer
Now, we can use these custom layers to build the Keras model
model summary & plot
Train the Keras model with custom layers for 3 epochs
Saving custom objects model with HDF5 format
Saving custom objects model with the TensorFlow SavedModel format
Fix the error occurred when saving custom objects with HDF5 format
Conclusions


We reviewed the related concepts to save and load a Keras model with custom objects in TensorFlow.
We have implemented custom objects (layers) in a Keras model.
We saved the entire trained model with 2 different formats: TensorFlow SavedModel & HDF5.
We loaded the saved model successfully and use it for prediction.
If you do not have a specific reason, I suggest you to use TensorFlow SavedModel format especially when the model has some custom objects.
You can learn more about TensorFlow & Keras layers in this and that playlists on Murat Karakaya Akademi YouTube channel.
As suggested by the official TensorFlow documents, we need to add get_config() method to the custom objects!
According to the TensorFlow Core Tutorials: Saving custom objects with HDF5 format

To save custom objects to HDF5, you must do the following:

Define a get_config() method in your object, and optionally a from_config class method.
get_config(self) returns a JSON-serializable dictionary of parameters needed to recreate the object.
from_config(cls, config) uses the returned config from get_config to create a new object. By default, this function will use the config as initialization kwargs (return cls(**config)).
Pass the object to the custom_objects argument when loading the model. The argument must be a dictionary mapping the string class name to the Python class. E.g.

tf.keras.models.load_model(path, custom_objects={'CustomLayer': CustomLayer})

Note that: In the above custom layer we have not implemented get_config() method. THerefore, if we try to save the trained model with the custom layers, we will get the following error messages
if you only supply the name of the saved model, e.g. model.save('my_model'), then the saved model format will be TF savedmodel as default.

If you supply the H5 file extension, e.g. model.save('my_model.h5'), then the saved model format will be HDF5.
he H5 format does not include two things that are saved by the TF SavedModel format:

External losses & metrics added via model.add_loss() & model.add_metric() are not saved(unlike SavedModel).

If you have such losses & metrics on your model and you want to resume training, you need to add these losses back yourself after loading the model. Note that this does not apply to losses/metrics created inside layers via self.add_loss() & self.add_metric(). As long as the layer gets loaded, these losses & metrics are kept, since they are part of the call method of the layer.

The computation graph of custom objects such as custom layers is not included in the saved file. At loading time, Keras will need access to the Python classes/functions of these objects in order to reconstruct the model. See Custom objects.

The buttom line is that "the SavedModel format saves and loads custom layers without the original class definition"

If you want to learn more about this property, look for the save_traces parameter.

I will show below how to save models with custom objects when saved in the H5 format.
What are the Save Formats in TensorFlow / Keras?
We have 2 model save formats:

Either Tensorflow SavedModel or
HDF5
Defaults:

for TF 2.X is Tensorflow SavedModel
for TF 1.X is HDF5