--Share how to easily create an image recognition model and easily run it as an app
--google colaboratory (runtime: GPU) (tensorflow 2.0) (google Chrome)
-[Dataset creation method for image recognition model](#Dataset creation method for image recognition model) -[How to create an image recognition model](#How to create an image recognition model) -[How to load and move the created image recognition model in Android Studio](#How to load and move the created image recognition model in android-studio) -[My failure story](# My failure story)
Take the following steps
Skip if you have already collected images If you haven't collected it yet, google_image_download (reference URL) is convenient. It downloads images by specifying keywords, extensions, sizes, number of sheets, etc.
Divide the images into good feelings for training and testing (80% training: about 20% test?) Specifically, create a folder with the following structure images ├train │└A │ └a01.jpg │ └aslfdjk.png │ ... │└B │ └ba.jpg │ └dskrup.png │ ... │└C │ └ba.jpg │ └sdddrrd.png │ ... │ ├validation │└A │ └fwwqd.jpg │ └qiita.png │ ... │└B │ └sddd.jpg │ └reag.png │ ... │└C │ └vtet.jpg │ └fhyr.png │ ...
Create the model in the following steps
Focusing on simplicity, the code that diverted the tutorial provided by tensorflow is [(reference URL) * google colaboratory opens suddenly when you press the URL](https://colab.research.google.com/drive/ 1p-MY8C_H238TWffWrRjC8cc20alZACSD) For how to use google colaboratory, refer to Official HP.
This part of the source code
from google.colab import drive
drive.mount('/content/drive')
PATH = '/content/drive/'+'Path to where your dataset is on google drive'
If you enter this, do you want to allow access to google drive? You will be asked to certify that. I personally don't care about it, so I authenticated without thinking about it, but if you care about it, you should stop loading your own dataset with google colaboraotry.
This part of the source code
After that, you can train the model by executing the following code cells in sequence.
history = model.fit_generator(
train_data_gen,
steps_per_epoch=total_train // batch_size,
epochs=epochs,
validation_data=val_data_gen,
validation_steps=total_val // batch_size
)
only this
saved_model_dir = base_dir+'Where you want to save'
tf.saved_model.save(model, saved_model_dir)
Take the following steps
only this
converter = tf.lite.TFLiteConverter.from_saved_model(saved_model_dir)
tflite_model = converter.convert()
with open(base_dir+'Model name.tflite', 'wb') as f:
f.write(tflite_model)
Download the saved model and save it locally.
First, install Android studio (reference URL), As usual, the quick start provided by tensorflow is used. Reference URL If you follow the steps, a folder named example will be created in the local environment, so open the android project in the following path. \examples\lite\examples\image_classification\android
Follow the steps below
Place it in the following folder. \examples\lite\examples\image_classification\android\app\src\main\assets There are some .tflite in the same place.
In this case, write classes A, B, and C as follows and save them in the same location as the .tflite file.
mylabel.text
A
B
C
First, \examples\lite\examples\image_classification\android\app\src\main\java\org\tensorflow\lite\examples\classification\tflite\ClassifierFloatMobileNet.java 56th line
return "mobilenet_v1_1.0_224.tflite";
Is rewritten as follows
return "Model name.tflite";
next, \examples\lite\examples\image_classification\android\app\src\main\java\org\tensorflow\lite\examples\classification\tflite\Classifier.java 110th line
public static Classifier create(Activity activity, Model model, Device device, int numThreads)
throws IOException {
if (model == Model.QUANTIZED) {
return new ClassifierQuantizedMobileNet(activity, device, numThreads);
} else {
return new ClassifierFloatMobileNet(activity, device, numThreads);
}
}
Is rewritten as follows
public static Classifier create(Activity activity, Model model, Device device, int numThreads)
throws IOException {
//if (model == Model.QUANTIZED) {
//return new ClassifierQuantizedMobileNet(activity, device, numThreads);
//} else {
return new ClassifierFloatMobileNet(activity, device, numThreads);
//}
}
Finally, run a make project in Android Studio. After that, connect the smartphone to the PC and execute Run'app'. Then, an application named TFL Classify will be installed and executed on your smartphone. Confidence of A, B, and C (indicating how likely the class is in%) is displayed at the bottom of the screen.
I will share some of the failure (?) Stories that the author stumbled upon that may be helpful.
I tried to convert the pytorch model to the tensorflow LITE model and ended up frustrated. I mainly referred to these two websites, URL1, [URL2](https://heartbeat.fritz.ai/deploying-pytorch-and -keras-models-to-android-with-tensorflow-mobile-a16a1fb83f2) Both of them gave me an unclear error and stopped. When I think about it now, it may have been caused by the existence of both keras and tf.keras. Shall we verify it next time?
This time, I wanted to move my own model with the android application as soon as possible, so I ended up creating a model using the tensorflow tutorial and converted the keras model to the tensorflow LITE model as described above.
As described in [URL1] above (https://qiita.com/lain21/items/9f9f9707ebad4bbc627d)
Meanwhile, TensorFlow's clear advantages over PyTorch include:
Can learn using TPU Rich APIs that facilitate front-end deployment, such as TFLite and TensorFlow.js
It seems that tensorflow is currently more supportive for facilitating conversion to apps.
While collecting various codes, keras and tf.keras were mixed. From the middle, I rewrote it to tf.keras as a man who absolutely erases keras.
I don't notice that the output of the model is one in the default tutorial code, and there is not enough output in android studio! I made an error saying. I thought that the cause of the error was to create a .tflite file from keras using TFLiteConverter, so I spent a lot of time there ... After all, at this URL, while trying to evaluate the self-made model, it became "something wrong with the output shape?" I noticed that the shape of
I tried to quantize the model, but I gave up because it could not be read by android studio. After all, I used the float model. I tried it because the size of the model would be smaller and the processing would be faster if it was quantized, but I couldn't. The cause of this is still unknown.
At first, I hadn't decided what to use for object recognition, so I was thinking of using yolo on darknet, but I didn't know how to use my own model, so I gave up because I didn't know what to do. From an acquaintance, pytorch is easy! It led to failure 1. Don't give up as soon as you think this way. .. ..
You can easily collect images using google_image_download with [Collect images of classes A, B, C](# Collect images of classes A, B, C)! I wrote that, but this is quite difficult. Of course, inappropriate images are also downloaded, so it is difficult to throw them away or crop them, and then the work of classifying is waiting. I understand why annotation is a job in the world, but I don't want to do much because real MP is likely to decrease. .. .. There is an active learning as a related word, and it seems to be a method of automatically selecting an image that increases the recognition rate. It would be convenient if possible. I wonder?
After I started writing this article, I noticed that the contents of the tensorflow tutorial had changed. Actually, I used the transfer learning tutorial, but since it was troublesome to publish my code, I had no choice but to use the image classification tutorial for explanation. I think transfer learning, which is said to improve accuracy with a small number of data, is more convenient than creating a model by yourself. The current transfer learning tutorial is designed to read data from tf's, and it is complicated to change and write it, so I gave up using it for explanation. I want to use my data for transfer learning! If there is a request, you may feel like doing your best. .. ..
Recommended Posts