[PYTHON] Image recognition of garbage with Edge (Raspberry Pi) from zero knowledge using AutoML Vsion and TPU

It is a memorandum I will write about how to do it

About the author

1st year master's degree at a certain university Studying video coding, electrical circuits, and machine learning I have touched C, C ++, JAVA, python, chainer, Ubuntu, etc. Totally ignorant about Raspberry Pi, GCP, API, Raspbian

environment

Ubuntu 18.04.2 LTS (Nopa) (Windows is OK) Raspbian（Raspberry Pi 3 Model B+） Coral accelerator EDGE TPU

things to do

Real-time image recognition of dust using a USB camera and TPU with Raspberry Pi (this article) Create a trash can that can be automatically identified by controlling hardware using the result (outside this article)

reference

Read and save image files with Python, OpenCV (imread, imwrite) Operating Raspberry Pi on Windows (SSH) I tried using the standard Windows 10 SSH client Try a custom model of AutoML Vision Edge with Coral Edge TPU Try a custom model of AutoML Vision Edge on a Coral Edge TPU (Try a custom model of AutoML Vision Edge on a Coral Edge TPU) How to start machine learning! An easy introduction with Google's AutoML Vision Edge

table of contents

Preparing the Raspberry Pi

1. Insert the OS
2. SSH settings
3. Preparation for doing AutoML with Raspberry Pi
4. Install OpenCV
5. Preparing to connect the USB camera

Creating a dataset

1. Install OpenCV
2. Create and run a photographing program
3. Compress to ZIP

Preparing AutoML in the browser

1. Data set registration
2. Training
3. Model download

Creating a program

1. Create a program (function) to take a picture
2. Create a program (function) that recognizes images based on photos

Preparing the Raspberry Pi

Put the OS

Download the OS from the Official Site, insert it into the SD card, insert it into the Raspberry Pi, and start it. If you have trouble changing the directory from Japanese to English, it is recommended to install the OS in English from the beginning. If you can't, you may find this article helpful: [Notes on installing a recent Raspberry Pi image (Raspbian)](https://www.1ft-seabass.jp/memo/2018/07/23/raspbian- install-201807-memo /)

SSH (remote connection) settings

Launch terminal with ctrl + alt + t on Raspberry Pi sudo raspi-config Type a command from the open window 8 Advanced Options A4 SSH Enable Ssh is enabled on

Start terminal with ctrl + alt + t on Ubuntu After connecting to the same wifi as Raspberry Pi Connect with ssh [email protected] The initial password is It's raspberry

You can also connect with ssh [ip address] The ip address can be found on the Raspberry Pi with the ʻip acommand Look at the tokoro of wlan0inet 192.168.195.212/24 brd 192.168.195.255 scope global noprefixroute wlan0 If there is such a thing, 192.168.195.212 is the IP address So thessh 192.168.195.212` command is OK For details, see ifconfig is deprecated !? How to find out the IP address of Raspberry Pi without using the ifconfig command

You can do the same with windows like this

Preparing to do AutoML with Raspberry Pi

Reference: Try a custom model of AutoML Vision Edge with Coral Edge TPU Module import

cd ~/
wget --no-check-certificate https://dl.google.com/coral/edgetpu_api/edgetpu_api_latest.tar.gz -O edgetpu_api.tar.gz --trust-server-names
tar xzf edgetpu_api.tar.gz
cd edgetpu_api
bash ./install.sh

If I didn't enter --no-check-certificate, I got an error, so I included it.

Creating a dataset

OpenCV installation

An error occurred when I put opencv in python3 with Raspberry Pi [Corrective action] Create and run a photo-taking program Compress to ZIP

Preparing AutoML in the browser

Data set registration

training Model download

• This method has failed. I will write about the background. In addition, if you install the Street here SDK from the beginning, there will probably be no problem. Follow the guide 1. Export your model to Google Cloud Storage. $ gsutil cp -r gs://dust2/dust_box_model/ ./download_dir When I typed in, I was told Command'gsutil' not found Install with sudo apt install gsutil and retry ← This is a mistake

gsutil cp -r gs://＊＊＊＊＊/ ./download_dir
No command was given.

Choose one of -b, -d, -e, or -r to do something.
Try `/usr/bin/gsutil --help' for more information.

Will be said. Q. Is there an error even though the -rʻoption is properly attached? ?? ?? A. It seems that you have installed a different gsutil` from google Download from from here If you do this street, you can probably go

I can't do anything well, so I will try again from Raspberry Pi

First of all, if the time zone is off, it will be played sudo date -s '11/04 20:10 2019' Set the time with Since the version of python was 2 series, it is as it is curl https://sdk.cloud.google.com | bash ↑ This command often gives a 403 error in my environment, but I didn't give up and executed it several times and it was the 4th time.

 $ exec -l $SHELL
 $ which gcloud
 $ gcloud init

I couldn't do it well because I couldn't open the browser with SSH connection, so I ran it directly with Raspberry Pi. By the way, @ was input by Shift + 2.

Import the Google cloud SDK

#Setting environment variables
export CLOUD_SDK_REPO="cloud-sdk-$(lsb_release -c -s)"

#Added cloud SDK URL
echo "deb http://packages.cloud.google.com/apt $CLOUD_SDK_REPO main" | sudo tee -a /etc/apt/sources.list.d/google-cloud-sdk.list

#Import public key
curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -

#Update and install SDK
#I often get download errors--fix-missing is added
sudo apt-get update && sudo apt-get install google-cloud-sdk --fix-missing

#By the way, install the python component
sudo apt-get install google-cloud-sdk-app-engine-python

#Start command
gcloud init

After this Vision $ gsutil cp -r gs://****/*****/your_model/ ./download_dir Except for one of the model and this / from ← here is important $ gsutil cp -r gs://****/*****/your_model ./download_dir I was able to download it at. How hard is it to do a one-line command? .. .. ..

Run TPU

Download the program to run the TPU git clone https://github.com/google-coral/tflite.git

I was told that there is no tflite_runtime so I installed it Download the file from here Those who think the link is suspicious are correct. For those people, please go to Source In the directory where this file is located pip3 install tflite_runtime-1.14.0-cp37-cp37m-linux_armv7l.whl Can be installed with a command

afterwards python3 classify_image.py --m edgetpu_model.tflite --l dict.txt --i images.jpeg Can be executed with Match each file to your own file path

Creating a program

Create a program (function) to take a picture

Create a program (function) that recognizes images based on photos

Added the following functions to classify_image.py I changed the command line arguments to normal arguments, please tell me if there is a better way

def get_label(labels = "dict.txt",model = "edgetpu_model.tflite",input_i = "images.jpeg ",count =5,threshold = 0.0,top_k =1):

  labels = load_labels(labels) if labels else {}

  interpreter = make_interpreter(model)
  interpreter.allocate_tensors()

  size = classify.input_size(interpreter)
  image = Image.open(input_i).convert('RGB').resize(size, Image.ANTIALIAS)
  classify.set_input(interpreter, image)

  for _ in range(count):
    start = time.monotonic()
    interpreter.invoke()
    inference_time = time.monotonic() - start
    classes = classify.get_output(interpreter, top_k, threshold)

  for klass in classes:

    label = (labels.get(klass.id, klass.id))
    score = klass.score

    return label,score

Then, you can get the label and reliability by importing and executing as follows.

import classify_image as clfi

if __name__ == '__main__':

	labels ="dict.txt"
	model = "edgetpu_model.tflite"
	input_i ="images.jpeg "
	label ,score = clfi.get_label(labels =labels,model = model , input_i =input_i)
	print(label)
	print(score)

You can get the result like this

INFO: Initialized TensorFlow Lite runtime.
burnable
0.703125

For the time being, learn with about 30 data for each label, I tested it with burning garbage picked up from the net, but it's surprising. Well, it's not a level that I can comment on as accuracy yet ... I'm stocking garbage, so I'd like to do it properly when the data increases.

After all, the screen is displayed on the Raspberry Pi for development, so the heat generation is amazing.