[PYTHON] Implement TensorFlow Lite on Mac [2019 Edition]

I will explain the flow until running TensorFlow Lite on Mac.

environment

The environment whose operation has been confirmed is as follows. · MacOS Catalina version 10.15 -Python 3.7.4 ・ Conda 4.7.12 ・ TensorFlow 1.15.0 ・ Keras 2.2.4

Environment

Install Anaconda from the following URL https://www.anaconda.com/distribution/ Double-click the downloaded installer package to start it. Agree to the terms of use, decide the save destination, and install. Launch Jupyter Notebook from Home. Decide where you want to work and create an ipynb file with NEW → Python3. When you open the file, you will be taken to the screen where you can write a program like this. I will write the program here. Next, install the package for running TensorFlow. Create a new environment by selecting "Enviroments" → "Create". Not required when using base (root). Change the search criteria to "Not installed" and search for "tensorflow". Then select "keras" and "tensorflow" of the package that appears and click Apply. Start Jupyter as before and write and execute the program as follows. Execution is "control + Enter" or "Shift + Enter". You can see that there are no errors. The usage is roughly like this.

Postscript Also have nomkl, matplotlib and pillow installed. nomkl seems to prevent the kernel from dying when running tensorflow. matplotlib is used to display images. The pillow is used to load the image.

Build a model

In order to generate a model of TensorFlow Lite, you first need to create a model of TensorFlow. This time we will use a dataset called cifar10. https://www.cs.toronto.edu/~kriz/cifar.html cifar10 is a dataset labeled with 60,000 images. It is divided into airplanes, cars, birds, cats, deer, dogs, frogs, horses, ships and trucks. We will learn this and create a model that can classify images.

Below is the code for learning the image. Since the number of epochs is set to 20, it takes a long time.

"""
Import required libraries and pre-process images
"""
from keras.models import Sequential
from keras.layers.convolutional import Conv2D
from keras.layers.pooling import MaxPool2D
from keras.layers.core import Dense,Activation,Dropout,Flatten
from keras.datasets import cifar10
from keras.utils import np_utils

 #Download cifar10
(x_train,y_train),(x_test,y_test)=cifar10.load_data()

#Image 0-Normalize in the range of 1
x_train=x_train.astype('float32')/255.0
x_test=x_test.astype('float32')/255.0

#One correct label-Convert to Hot expression
y_train=np_utils.to_categorical(y_train,10)
y_test=np_utils.to_categorical(y_test,10)

"""
Build a model for TensorFlow
"""
model=Sequential()

model.add(Conv2D(32,(3,3),padding='same',input_shape=(32,32,3)))
model.add(Activation('relu'))
model.add(Conv2D(32,(3,3),padding='same'))
model.add(Activation('relu'))
model.add(MaxPool2D(pool_size=(2,2)))
model.add(Dropout(0.25))

model.add(Conv2D(64,(3,3),padding='same'))
model.add(Activation('relu'))
model.add(Conv2D(64,(3,3),padding='same'))
model.add(Activation('relu'))
model.add(MaxPool2D(pool_size=(2,2)))
model.add(Dropout(0.25))

model.add(Flatten())
model.add(Dense(512))
model.add(Activation('relu'))
model.add(Dropout(0.5))
model.add(Dense(10,activation='softmax'))

model.compile(optimizer='adam',loss='categorical_crossentropy',metrics=['accuracy'])

history=model.fit(x_train,y_train,batch_size=128,nb_epoch=20,verbose=1,validation_split=0.1)

#Save model and weight
json_string=model.to_json()
open('cifar10_cnn.json',"w").write(json_string)
model.save_weights('cifar10_cnn.h5')
model.save('cifar10_cnn_model.h5')

#Model display
model.summary()

#Evaluation
score=model.evaluate(x_test,y_test,verbose=0)
print('Test loss:',score[0])
print('Test accuracy:',score[1])

When you run I think that the files "cifar10_cnn.h5" and "cifar10_cnn_model.h5" have been generated. "Cifar10_cnn.h5" stores only the model weights, and "cifar10_cnn_model.h5" stores the model structure and weights. The accuracy was 78%. It seems that the accuracy can be improved by referring to the following site.

Accuracy 95% with CIFAR-10-Techniques to improve accuracy with CNN- Achieve 90% of CIFAR-10 Validation Accuracy with 10-layer convolutional neural network

Actually classify images

Image classification will be performed using the saved model. First, prepare an image to predict.

Create a folder according to the hierarchy written in the code, put the image in it, and execute the following code.

"""
Predict using the images you picked up
"""
from keras.models import model_from_json
import matplotlib.pyplot as plt
from keras.preprocessing.image import img_to_array, load_img
from tensorflow.python.keras.models import load_model

#Image loading
temp_img=load_img("./images/airplane1.jpeg ",target_size=(32,32))

#Convert image to array 0-Normalize with 1
temp_img_array=img_to_array(temp_img)
temp_img_array=temp_img_array.astype('float32')/255.0
temp_img_array=temp_img_array.reshape((1,32,32,3))

#Load trained models and weights
json_string=open('cifar10_cnn.json').read()
model=model_from_json(json_string)
model.compile(optimizer='adam',loss='categorical_crossentropy',metrics=['accuracy'])
model.load_weights('cifar10_cnn.h5')
# model = load_model('cifar10_cnn_model.h5')

#View model
model.summary()

#Anticipate the image
img_pred=model.predict_classes(temp_img_array)
print('\npredict_classes=',img_pred)
print('model=',model)

plt.imshow(temp_img)
plt.title('pred:{}'.format(img_pred))
plt.show()

"""
0 - airplane
1 - automobile
2 - bird
3 - cat
4 - deer
5 - dog
6 - frog
7 - horse
8 - ship
9 - truck
"""

If all goes well, the image and index number will be output. ↑ Like this You can see that the index number and the image match. Image classification is successful.

Convert to model for TensorFlow Lite

Then, convert the model generated earlier for TensorFlow Lite.

#Existing Keras model(cifar10_cnn_model.h5)From the model for TensorFlow Lite(cifar10_cnn.tflite)Create

import tensorflow as tf

if __name__ == '__main__':
    converter = tf.lite.TFLiteConverter.from_keras_model_file("cifar10_cnn_model.h5")
    tflite_model = converter.convert()
    open("cifar10_cnn.tflite", "wb").write(tflite_model)

Executing this code will generate "cifar10_cnn.tflite". This is the model for TensorFlow Lite.

Let's use this model to classify images.

#Identify the genre from the input image using the model for TensorFlow Lite

import tensorflow as tf
import numpy as np
from keras.models import model_from_json
import matplotlib.pyplot as plt
from keras.preprocessing.image import img_to_array, load_img
from tensorflow.python.keras.models import load_model

if __name__ == '__main__':
    # prepara input image
    #Image loading
    temp_img=load_img("./images/dog1.jpeg ",target_size=(32,32))

    #Convert image to array 0-Normalize with 1
    temp_img_array=img_to_array(temp_img)
    img=temp_img_array.astype('float32')/255.0
    img=temp_img_array.reshape((1,32,32,3))

    # load model
    interpreter = tf.lite.Interpreter(model_path="cifar10_cnn.tflite")
    interpreter.allocate_tensors()
    input_details = interpreter.get_input_details()
    output_details = interpreter.get_output_details()

    # set input tensor
    interpreter.set_tensor(input_details[0]['index'], img)

    # run
    interpreter.invoke()

    # get outpu tensor
    probs = interpreter.get_tensor(output_details[0]['index'])

    # print result
    result = np.argmax(probs[0])
    score = probs[0][result]
    print("Predicted image index:{} [{:.2f}]".format(result, score)) 
    
    plt.imshow(temp_img)
    plt.title('pred:{}'.format(img_pred))
    plt.show()

"""
0 - airplane
1 - automobile
2 - bird
3 - cat
4 - deer
5 - dog
6 - frog
7 - horse
8 - ship
9 - truck
"""

If this also works, the image and index number will be output.

Comparison of Tensorflow and Tensorflow Lite

The following comparison depends on the image data, so please use it as a reference only. We measured each time it took to predict the image.

Model for keras
Elapsed time: 0.8839559555053711
Elapsed time: 0.6288352012634277
Elapsed time: 0.5877768993377686
Elapsed time: 0.5789699554443359
Elapsed time: 0.5908827781677246
Elapsed time: 0.7207329273223877
Elapsed time: 0.7104830741882324
Elapsed time: 0.6035618782043457
Elapsed time: 0.5244758129119873
Elapsed time: 0.5348677635192871
Average elapsed time: 0.636454225

Model for Tensorflow Lite
Elapsed time: 0.27948904037475586
Elapsed time: 0.05380606651306152
Elapsed time: 0.022572994232177734
Elapsed time: 0.06809115409851074
Elapsed time: 0.07050800323486328
Elapsed time: 0.06940007209777832
Elapsed time: 0.12052798271179199
Elapsed time: 0.17615199089050293
Elapsed time: 0.12544798851013184
Elapsed time: 0.027255773544311523
Average elapsed time: 0.101325107

Tensorflow Lite is faster, isn't it? However, there is a difference in accuracy.

Cat was the only unpredictable image in the keras model. On the other hand, in the Tensorflow Lite model, airplane, Bird, cat, and Frog could not be predicted. The model converted for Tensorflow Lite seems to be considerably less accurate.

When run on the MNIST model with 99.4% accuracy, it was equally predictable for keras and Tensorflow Lite. When using TensorflowLite, it seems necessary to prepare a model with fairly high accuracy.

reference

Official sample of Tensorflow Lite not friendly to beginners Let's build CNN with Keras and classify images of CIFAR-10