I created a temperature prediction model of tensorflow lite that runs on edgetpu (coral) connected to Raspberry pi4. The Tensorflow version is 2.3.1, the edgetpu compiler version is 15.0.340273435, and the edgetpu runtime version is 2.14.1.
Create and train a 1D CNN-based model using Tensorflow Keras on Google Colab
After training, quantize the model and convert it to a tensorflowlite model
Compile with edgetpu compiler to work with edgetpu
Predict using test data on Raspberry pi4 & predict CPU temperature of Raspberry pi4
Using the Japan Meteorological Agency site where you can search past weather data, one day at a certain place We have collected data on average temperature for about 20 years.
I created a one-dimensional CNN-based model using the functional api of Keras that uses the average temperature of 30 days as input and predicts the average temperature of the next day. ..
inputs = tf.keras.Input(shape=(30,1))
cnn1 = tf.keras.layers.Conv1D(filters=1,kernel_size=10,strides=1,activation='relu',input_shape=(30,1))
cnn2 = tf.keras.layers.Conv1D(filters=1,kernel_size=5,strides=1,activation='relu')
cnn3 = tf.keras.layers.Conv1D(filters=1,kernel_size=3,strides=1,activation='relu')
dense1 = tf.keras.layers.Dense(units=8,activation='relu')
dense2 = tf.keras.layers.Dense(units=1)
x = cnn1(inputs)
x = cnn2(x)
x = cnn3(x)
x = tf.keras.layers.Flatten()(x)
x = dense1(x)
outputs = dense2(x)
model = tf.keras.Model(inputs=inputs,outputs=outputs)
model.compile(optimizer="Adam",loss="mean_squared_error",
metrics="binary_accuracy")
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_14 (InputLayer) [(None, 30, 1)] 0
_________________________________________________________________
conv1d_31 (Conv1D) (None, 21, 1) 11
_________________________________________________________________
conv1d_32 (Conv1D) (None, 17, 1) 6
_________________________________________________________________
conv1d_33 (Conv1D) (None, 15, 1) 4
_________________________________________________________________
flatten_5 (Flatten) (None, 15) 0
_________________________________________________________________
dense_24 (Dense) (None, 8) 128
_________________________________________________________________
dense_25 (Dense) (None, 1) 9
=================================================================
Total params: 158
Trainable params: 158
Non-trainable params: 0
_________________________________________________________________
"""
I trained with a batch size of 50 and an epoch number of 30.
model.fit(x=in_temp,y=out_temp,batch_size=50,epochs=30)
To run on edgetpu, it needs to be quantized to an 8-bit integer. Since the input size needs to be fixed for quantization, specify `` `sigantures``` and save the model once. Here, the input size is set to 1x30x1.
opt = tf.function(lambda x:model(x))
BACTH_SIZE = 1
STEPS = 30
INPUT_SIZE = 1
concrete_func = opt.get_concrete_function(tf.TensorSpec([BACTH_SIZE,STEPS,INPUT_SIZE],
model.inputs[0].dtype,name="inputs")
)
model.save('/content/weather',save_format="tf",signatures=concrete_func)
Then quantize the model. See Tensorflow's Documentation (https://www.tensorflow.org/lite/performance/post_training_integer_quant) for more information.
in_tempIs the numpy ndarray you entered when training the model.
conv_data = in_temp[0]
conv_data = conv_data.reshape(1,30,1)
def representative_dataset_gen():
for i in range(len(conv_data)):
yield [conv_data[i]]
converter_edgetpu = tf.lite.TFLiteConverter.from_saved_model("/content/weather")
converter_edgetpu.optimizations = [tf.lite.Optimize.DEFAULT]
converter_edgetpu.representative_dataset = representative_dataset_gen
converter_edgetpu.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
converter_edgetpu.inference_input_type = tf.uint8
converter_edgetpu.inference_output_type = tf.uint8
converter_edgetpu.experimental_new_converter = True
tflite = converter_edgetpu.convert()
Finally save the model.
open("cnn_weather_lite_quantized.tflite","wb").write(tflite)
Install the edgetpu compiler on Google Colab. Please refer to Documentation for the details of the installation method.
Directly because the CPU architecture requires x86-64 as shown in Documentation and the CPU architecture of Raspberry pi4B is ARMv8. I don't think it can be installed on Raspberry pi 4. (CPU architecture of Raspberry pi: https://nw-electric.way-nifty.com/blog/2020/02/post-6c09ad.html)
!curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
!echo "deb https://packages.cloud.google.com/apt coral-edgetpu-stable main" | sudo tee /etc/apt/sources.list.d/coral-edgetpu.list
!sudo apt -y update
!sudo apt-get install edgetpu-compiler
Reference: https://coral.ai/docs/edgetpu/compiler/#usage
!edgetpu_compiler /content/cnn_weather_lite_quantized.tflite
From the Japan Meteorological Agency site, we have prepared data for 30 days of average daily temperature again. Then I ran the following code (when using edgetpu) on the Raspberry pi to predict the temperature for the next day. We also measured the execution time and compared the time with the CPU usage. The inference code was based on the "Load and run model in Python" section of the Tensorflow site. It will be translated into Japanese automatically, so if the translation is incorrect, we recommend reading it in English.
import numpy as np
import pandas as pd
import tflite_runtime.interpreter as tflite
import time
def main(args):
interpreter = tflite.Interpreter('/home/pi/cnn_weather/cnn_weather_lite_quantized_edgetpu.tflite',
experimental_delegates=[tflite.load_delegate('libedgetpu.so.1')])
data = pd.read_csv('/home/pi/cnn_weather/test.csv')
test_data = np.asarray(data.loc[:len(data),"Average temperature"],dtype=np.uint8)
test_data = test_data.reshape(1,30,1)
start = time.perf_counter()
interpreter.allocate_tensors()
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()
input_shape = input_details[0]['shape']
interpreter.set_tensor(input_details[0]['index'],test_data)
interpreter.invoke()
output_data = interpreter.get_tensor(output_details[0]['index'])
end = time.perf_counter()
print("The next day's temperature is " + str(output_data[0,0]) + " degrees Celsius.")
print("It took " + str((end-start)*1000) + " ms.")
return 0
if __name__ == '__main__':
import sys
sys.exit(main(sys.argv))
The result is shown. The top is the result of executing with edgetpu, the middle is the result of executing the unquantized model on the CPU, and the bottom is the result of executing the quantized model on the CPU. Quantization makes it faster, but using edgetpu results in slower ... Since the execution time is 2ms, it may take longer to call edgetpu etc. using edgetpu. By the way, you can see that the accuracy drops considerably when quantized. (The test data is from the end of September to the end of October, and since it starts at about 22 ° C and ends at about 18 ° C, I feel that 24 ° C and 26 ° C are quite off the mark.)
pi@raspberrypi:~$ python3 /home/pi/cnn_weather/edgetpu_time.py
The next day's temperature is 24 degrees Celsius.
It took 2.218683000137389 ms.
pi@raspberrypi:~$ python3 /home/pi/cnn_weather/cpu_time.py
The next day's temperature is 17.671713 degrees Celsius.
It took 3.6856149999948684 ms.
pi@raspberrypi:~$ python3 /home/pi/cnn_weather/cpu_quantized_time.py
The next day's temperature is 26 degrees Celsius.
It took 1.4244879994294024 ms.
I'm running it on a small device called Raspberry pi, so I wanted to try something like real-time prediction. I wanted to predict it based on the outside temperature, but it seems difficult to make it myself because there is no sensor, so I tried to predict it based on the CPU temperature.
I took the CPU temperature with $ cat / sys / class / thermal / thermal_zone0 / temp every second, and predicted the temperature after 1 second when 30 seconds were gathered.
In addition, $ cat / sys / class / thermal / thermal_zone0 / temp returns the value of CPU temperature multiplied by 1000, so I used it by dividing the output result by 1000.
Below is the code I ran on my Raspberry pi. (Only the code when using edgetpu is shown)
import numpy as np
import tflite_runtime.interpreter as tflite
import time
import subprocess
def main(args):
start = time.perf_counter()
interpreter = tflite.Interpreter('/home/pi/cnn_weather/cnn_weather_lite_quantized_edgetpu.tflite',
experimental_delegates=[tflite.load_delegate('libedgetpu.so.1')])
data = list()
for i in range(30):
res = subprocess.run(['cat', '/sys/class/thermal/thermal_zone0/temp'],
stdout=subprocess.PIPE)
get_start = time.perf_counter()
result = res.stdout.decode('utf-8')
result = int(result)/1000
data.append(result)
print(result,end='℃ ')
if (i+1)%10 == 0:
print()
get_end = time.perf_counter()
get_time = get_end-get_start
if get_time < 1:
time.sleep(1-get_time)
else:
print("Took " + str(get_time) + " s to get " + str(i) + "'s temp.")
pre_start = time.perf_counter()
np_data = np.asarray(data,dtype=np.uint8).reshape(1,30,1)
interpreter.allocate_tensors()
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()
input_shape = input_details[0]['shape']
interpreter.set_tensor(input_details[0]['index'],np_data)
interpreter.invoke()
pred = interpreter.get_tensor(output_details[0]['index'])
pre_end = time.perf_counter()
pre_time = pre_end - pre_start
if pre_time < 1:
print("The cpu's temp will be " + str(pred[0,0]) + "℃ in " +
str(1-pre_time) + " s.")
time.sleep(1-pre_time)
res = subprocess.run(['cat', '/sys/class/thermal/thermal_zone0/temp'],
stdout=subprocess.PIPE)
result = res.stdout.decode('utf-8')
result = int(result)/1000
print("The cpu's temp is " + str(result) + "℃.")
else:
print("The cpu's temp must have been " + str(pred[0,0]) + "℃ " +
str(1-pre_time) + " s ago.")
end = time.perf_counter()
print("Took " + str(end-start) + " s to run this code.")
return 0
if __name__ == '__main__':
import sys
sys.exit(main(sys.argv))
The result is shown.
The top is when using edgetpu, the middle is when using an unquantized model, and the bottom is when using a quantized model.
The listed temperatures are the CPU temperature data used for prediction. After all, the quantized model has an irrelevant output exceeding 100 ° C.
Also, as before, it takes time to use edgetpu. This time, I was measuring the time not only for inference but also for calling tf.lite.Interpreter, so that part may also take time by using edgetpu.
pi@raspberrypi:~$ python3 /home/pi/cnn_weather/predict_edgetpu.py
63.783℃ 64.757℃ 63.783℃ 63.783℃ 63.296℃ 62.809℃ 63.296℃ 63.296℃ 62.809℃ 62.809℃
62.809℃ 63.296℃ 62.322℃ 62.809℃ 63.783℃ 62.809℃ 63.783℃ 63.783℃ 62.322℃ 62.809℃
62.322℃ 63.783℃ 62.809℃ 62.322℃ 62.322℃ 62.322℃ 62.322℃ 62.322℃ 62.322℃ 63.296℃
The cpu's temp will be 105℃ in 0.9969898569997895 s.
The cpu's temp is 61.835℃.
Took 34.21252226499928 s to run this code.
pi@raspberrypi:~$ python3 /home/pi/cnn_weather/predict_cpu.py
63.783℃ 63.783℃ 63.296℃ 62.809℃ 63.783℃ 63.296℃ 62.809℃ 63.296℃ 62.809℃ 62.322℃
62.322℃ 62.322℃ 62.809℃ 62.322℃ 61.835℃ 62.322℃ 62.322℃ 61.348℃ 62.322℃ 62.322℃
63.296℃ 61.835℃ 62.322℃ 61.835℃ 61.348℃ 61.348℃ 61.835℃ 62.322℃ 62.809℃ 62.322℃
The cpu's temp will be 62.17556℃ in 0.9969654129999981 s.
The cpu's temp is 62.322℃.
Took 31.404364756001087 s to run this code.
pi@raspberrypi:~$ python3 /home/pi/cnn_weather/predict_cpu_quantized.py
63.296℃ 63.296℃ 62.809℃ 62.322℃ 62.322℃ 61.835℃ 61.835℃ 62.322℃ 61.835℃ 62.322℃
62.809℃ 62.322℃ 62.809℃ 62.322℃ 60.861℃ 62.322℃ 61.835℃ 61.835℃ 62.322℃ 61.835℃
61.835℃ 61.835℃ 61.348℃ 62.322℃ 60.861℃ 61.348℃ 62.322℃ 61.348℃ 61.835℃ 61.348℃
The cpu's temp will be 101℃ in 0.9984136980001495 s.
The cpu's temp is 61.835℃.
Took 31.43542323499969 s to run this code.
I created and ran a temperature prediction model of tensorflow lite that runs on edgetpu (coral) connected to Raspberry pi4. Quantization significantly reduced the accuracy (rather, it became a model that was almost meaningless), and the result was that the execution time was longer when using edgetpu than when using the CPU. This time, rather than creating a model with good accuracy, creating a model that infers using edgetpu was the goal for the time being, so I will end here for the time being. All of the data, code, and tensorflowlite model used this time can be found on github.
Thank you for watching so far. This is the first article, so please feel free to comment if you have any concerns or suggestions.
I was trying to do the same with LSTM before using 1D CNN, but I got an error like the one in here I got stuck, so I decided to use a 1D CNN. Looking at the issue linked to it, it seems that it is currently impossible to quantize RNN-based models such as LSTM. (The linked Tensorflow version is 2.2.0, this time I used 2.3.1)
Although it is not the same error statement, it seems that RNN systems such as LSTM do not support quantization even in Errors that seem to be related. I was asked if there was one. In addition, I was able to convert it to a tensorflow lite model and run it with Raspberry pi in the unquantized state.