[PYTHON] Difference and compatibility verification between keras and tf.keras # 1

End of development of Keras

It's been a while since I posted it. As you can see on github, the multi-backend Keras seems to have ended development in April 2020. It will be replaced by Keras in Tensorflow in the future. keras-team (For convenience, multi-backend Keras is referred to as mb-keras, and Tensorflow Keras is referred to as tf-keras) I would like to compare and verify mb-keras and tf-keras in this article.

• This time, I will leave it as # 1 and add articles as needed.

About mb-keras Tensorflow support

As mentioned on github, it seems that it supports up to Tensorflow 2.0. As of May 4, 2020, at the time of writing this article, Tensorflow's latest stabilizer is 2.1.0.

environment

In writing this article, I re-installed Tensorflow from CUDA etc., but I created a poor environment. mb-keras supports up to Python 3.6 ... (tensorflow supports up to Python 3.7) mb-keras 2.3.1 supports up to tensorflow 2.0, so this is good.

python -V
3.7.6

>>> import keras
>>> keras.__version__
'2.3.1'
>>> import tensorflow
>>> tensorflow.__version__
'2.0.0'

alias

It's basically the same, just with tensorflow in the head.

mb-keras

from keras.layers import Dense, Conv2D

tf-keras

from tensorflow.keras.layers import Dense, Conv2D

Object compatibility

Probably ** not **. However, there is no need for compatibility ... Check with the code below.

About Model and Layer

def mb_layers():
    from keras.layers import Input, Dense, Activation
    input_l = Input(shape=(64,))
    hidden = Dense(10)(input_l)
    output_l = Activation('softmax')(hidden)
    return input_l, output_l
def tf_layers():
    from tensorflow.keras.layers import Input, Dense, Activation
    input_l = Input(shape=(64,))
    hidden = Dense(10)(input_l)
    output_l = Activation('softmax')(hidden)
    return input_l, output_l

if __name__ == '__main__':
    mb_i, mb_o = mb_layers()
    tf_i, tf_o = tf_layers()
    
    from keras.models import Model as mbModel
    from tensorflow.keras.models import Model as tfModel
    
    # mb_in_tf = mbModel(tf_i, tf_o) #---- ※1
    # tf_in_mb = tfModel(mb_i, mb_o) #---- ※2
    mb_in_mb = mbModel(mb_i, mb_o)
    mb_in_mb.summary()               #---- ※3
    tf_in_tf = tfModel(tf_i, tf_o)
    tf_in_tf.summary()               #---- ※4

• 1 Run-time error occurs (put tf-keras Layer into mb-keras Model) TypeError: object of type 'Dense' has no len()
• 2 Run-time error occurs (Layer of mb-keras is inserted into Model of tf-keras) AttributeError: 'tuple' object has no attribute 'layer'
• 3 Execution result

stdout

Model: "model_7"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_23 (InputLayer)        (None, 64)                0         
_________________________________________________________________
dense_14 (Dense)             (None, 10)                650       
_________________________________________________________________
activation_12 (Activation)   (None, 10)                0         
=================================================================
Total params: 650
Trainable params: 650
Non-trainable params: 0
_________________________________________________________________

• 4 Execution result

stdout

Model: "model_2"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_24 (InputLayer)        [(None, 64)]              0         
_________________________________________________________________
dense_14 (Dense)             (None, 10)                650       
_________________________________________________________________
activation_11 (Activation)   (None, 10)                0         
=================================================================
Total params: 650
Trainable params: 650
Non-trainable params: 0
_________________________________________________________________

The output shape of the InputLayer is different, but can it be implemented as the same architecture? I want to check it from the next time onwards. For the time being, it seems that the making of Layer is different between mb-keras and tf-keras, so it is certain that it can not be thrown into Model.

About Optimizer

Optimizer can use tf-keras Optimizer for Model made with mb-keras.

model.compile

    from keras.optimizers import Adam as mbAdam
    from tensorflow.keras.optimizers import Adam as tfAdam
    mb_in_mb.compile(tfAdam(), 'mse')    #----※1
    tf_in_tf.compile(mbAdam(), 'mse')    #----※2

• 1 does not cause an error.
• 2 is the following error. ValueError: ('Could not interpret optimizer identifier:', <keras.optimizers.Adam object at 0x000001A4FC663F08>)

About usable layers

This seems to be quite different. ○ is implemented, × is not implemented. Note that not everything is a layer, as we just arranged the ones taken out with dir () almost as they are. Looking at the difference in Attention, is tf-keras more advanced in revision and abolition? (Override method and all lowercase letters are function-like, so I removed them from the list)

• Wrapper is implemented in mb-keras.

This summary

It turns out that mb-keras and tf-keras are almost incompatible. However, I don't feel the need.

As a case that seems to be a problem, mb-keras supports up to Python 3.6, so it will not be supported even if Python is updated in the future. On the other hand, the development of tf-keras will continue in the future. If you have created an AI system that uses mb-keras, you may be overwhelmed by the times, so you should immediately consider replacing it with tf-keras. As a concrete example that comes to mind first, ** Can a trained model created with mb-keras be loaded with tf-keras and inferred? **Toka Toka··· (I want to record after # 2 when I don't know when.)

Actually, I just built an in-house AI system with mb-keras before GW ... If you think that keras hasn't been updated recently, this is the place to go. I got it ...