Building a Raspberry Pi for studying Python and machine learning (RaspberryPi4 & Buster version (RaspberryPi3 is also possible))
[Details]
-** 2020/02/19 We reviewed some contents and confirmed the operation with Raspberry Pi 3 model B. ** ** -** Added Node-RED environment construction. ** **
Finally, the Raspberry Pi 4 model B is now available in Japan. I rewrote the previously posted [Building Raspberry Pi for Python and machine learning study] to the 4th edition of Raspberry Pi so that I can use it immediately. Besides the hardware, I have updated the version of Raspbian to the latest version (Buster 20200213). Also, it has been confirmed that the environment can be built with the same procedure with ** Raspberry Pi 3 model B **.
As of February 19, 2020, there are the following problems.
--The latest version of TensorFlow that can be installed with pip3 is 1.14.0 (as of 02/19/2020) --ʻImport tensorflow` I get an error when executing (can I ignore it?) -~~ Edge TPU is not officially supported (← supported by unofficial procedure) ~~ --~~ The latest version 4.1.1.26 of OpenCV that can be installed with pip3 does not work (← works if 4.1.0.25) ~~ -USB camera cannot be used with OpenCV that can be installed with ~~ pip3. (← Supported by apt install 3.2.0) ~~ -~~ There is no screen because I don't have a Micro HDMI cable ^^; (← Headless setup) ~~
[What you can do with the Raspberry Pi made in this procedure]
--Basic learning of Python --Data visualization (pandas, matplotlib)
- Computer Vision (OpenCV) --Machine learning (Scikit-Learn)
- DeepLearning (TensorFlow, Keras) --Inference using Coral Edge TPU --Implementing an app using the above (Flask)
- Jupyter Lab
- Node-RED
【System configuration】
--Raspberry Pi 3 or 4 Model B
- Raspbian Buster 20200213 --Coral Edge TPU (optional)
[0. Preparation of OS image]
I'm assuming you're using Raspbian Buster 20200213. As of February 19, 2020, you can download it from the official website, but I will write a mirror site just in case. [[Raspbian Buster 20200213 Mirror Site (ftp.jaist.ac.jp)]](http://ftp.jaist.ac.jp/pub/raspberrypi/raspbian/images/raspbian-2020-02-14/2020-02 -13-raspbian-buster.zip)
[1. Write and start OS image]
1-1. Writing OS image
Write the OS image prepared above to the SD card. Please refer to the following article for the procedure for writing to the SD card.
-[Set up Raspberry Pi Headless]
1-2. Creating a file for starting ssh
The current version of Raspbian has the ssh service disabled by default.
Create an empty file called ssh (without extension) in the" Boot "partition of the SD card so that it can boot automatically.
If you are using windows, the extension is hidden by default, so please check if there is an extension.
The ssh file is read and deleted when you start the Raspberry Pi.
1-3. Creating a configuration file for Wifi connection
As with ssh, if you put a file for Wifi settings in the "Boot" partition, it will be automatically read and written to the appropriate location.
Specifically, create a file named wpa_supplicant.conf in the" Boot "partition and describe the settings.
Please describe the following contents to connect to Wifi.
The settings need to be changed according to the access point to be connected.
wpa_supplicant.conf
ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
update_config=1
country=JP
network={
ssid="<SSID name>"
psk="<password>"
}
If you need to authenticate with the RADIUS server linked with Active Directory, please refer to the following article. [Connect from Raspeye to a wireless LAN that requires RADIUS authentication linked with AD (WPA2-EAP)]-Qiita
1-4. Start Raspberry Pi
When the above preparations are complete, remove the SD card from the PC, insert it into the Raspberry Pi body, and turn on the power.
[2. System Update & Network Settings]
2-1. System update
When the Raspberry Pi starts, connect with ssh or execute the following command in the terminal of the console to update the system. (It takes a few minutes)
System update
sudo apt update
sudo apt upgrade -y
sudo reboot
2-2. Disable ipv6 (optional)
It's not required, but it can get in the way in some cases, so disable it. This setting is not required for ipv6 operation.
ipv6 invalidation setting
sudo vi /etc/sysctl.conf
[Settings]
/etc/sysctl.conf
net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1
[Reflect settings]
Reflect settings
sudo sysctl -p
Or
sudo reboot
2-3. Fixed IP address (optional)
Fix the IP address if necessary.
This setting is not required when using in a DHCP environment.
Since this procedure assumes operation on a wireless LAN, the IP is fixedly set for the wireless LAN adapter or SSID.
Specifically, add the following contents to /etc/dhcpcd.conf.
Change <IP address>, <router address>, <DNS address>, and <your_ssid> according to your environment.
/etc/dhcpcd.conf
interface wlan0
inform <IP address>
static routers=<Router address>
static domain_name_servers=<DNS address>
noipv6
Or
SSID <your_ssid>
inform <IP address>
static routers=<Router address>
static domain_name_servers=<DNS address>
noipv6
When using in multiple wireless AP environments, it is convenient to fix with SSID.
2-4. Turn off the wireless LAN Power Management (optional)
This setting is not required when using in a wired LAN environment.
By default, the wireless LAN Power Management function is turned on, but this function may cause Wifi communication to become unstable.
Add a command to turn off Power Management to rc.local that is executed at startup.
rc.Editing local
sudo vi /etc/rc.local
rc.local
#Add the following before the exit statement and save
sudo iwconfig wlan0 power off
Reboot
sudo reboot
[3. Installation of required modules]
Install the modules required by the following operations. (Mainly used in OpenCV and Numpy)
Required module installation
sudo apt install -y libhdf5-dev libqtwebkit4 libqt4-test libatlas-base-dev libjasper-dev
[4. Installation related to Python3]
4-1. Installation of basic modules
Python3&pip3 installation
sudo apt install python3 python3-dev -y
This operation was not necessary because Buster was pre-installed.
4-1-1. Update of pip3 (optional)
** This step does not need to be performed **
If you follow the previous steps and update the pip3 command, it will throw an error and stop working when you run pip3.
Upgrade of pip3 [Failure]
#If you execute the following command, pip3 cannot be executed.
sudo pip3 install pip -U
pip3 -V
#If pip3 cannot be operated with the above command, recover with the following command
sudo python3 -m pip uninstall pip
sudo apt install python3-pip --reinstall
For the time being, it is possible to update the pip3 command by the following procedure, but it was not necessary in this procedure.
pip3 upgrade(option)
#Uninstall existing pip3 command
sudo apt remove python3-pip
#Install the latest pip3 command
wget https://bootstrap.pypa.io/get-pip.py
sudo python3 get-pip.py
pip3 -V
#Update setuptools
sudo pip3 install setuptools -U
4-2. Installation of various modules of Python3
Install various modules with the pip command. I have specified the versions of ~~ numpy and pandas, but since the latest binary package does not exist at the time of environment construction and the build starts and it takes time to install, the previous version is specified. Regarding TensorFlow, the latest one whose operation has been confirmed at the time of environment construction is specified. ~~ Each module specifies the latest version that can be package installed with Python 3.7 series as of February 19, 2020. Please change the version as needed.
Installation of various modules for Python3(2020/02/19 points)
# numpy
sudo pip3 install numpy==1.18.1
# scipy
sudo pip3 install scipy==1.4.1
# sklearn
sudo pip3 install scikit-learn==0.22.1
# matplotlib
sudo pip3 install matplotlib==3.1.3
# pandas
sudo pip3 install pandas==1.0.1
# seaborn
sudo pip3 install seaborn==0.10.0
# Tensorflow (2020/02/The latest is 1 as of 19.14.0)
sudo pip3 install tensorflow
#If you have updated pip3`--ignore-installed`Correspondence with options
#sudo pip3 install tensorflow --ignore-installed
# keras (tf.Not needed if you use keras)
# sudo pip3 install keras
# flask
sudo pip3 install flask flask_cors -U
# OpenCV
sudo pip3 install opencv-python==4.1.1.26
#Change and reflect environment variables(← I have to do this`import cv2`I get an error at runtime)
echo "export LD_PRELOAD=/usr/lib/arm-linux-gnueabihf/libatomic.so.1" | tee -a ~/.bashrc
source ~/.bashrc
# cv2.If you can't use multiple cameras with VideoCapture, install apt for provisional support.(However 3.2.0)
# sudo pip3 uninstall opencv-python
# sudo apt install python3-opencv
In addition, it is the most that pip is downloading the package, probably because "https://www.piwheels.org/" that manages the package recently is not working well. In many cases, the session will be disconnected during the process and the installation will fail as a result.
Re-execution often solves the problem, but large download sizes such as scipy and tensorflow are not very successful.
In that case, copy the URL of the package being downloaded that is displayed when you execute pip install, and download the file to your local folder with the wget command.
The wget command has a resume function even if the download fails, so it will automatically retry and somehow download the target file.
Then run sudo pip3 install <downloaded file> to install.
If you have a mirror site, you can point the reference to pip there, but I didn't know at the time of writing this article, so if anyone knows it, I would appreciate it if you could let me know.
5. Installation of Edge TPU
This item is not mandatory. Only run if you are using the Edge TPU.
Installing the library for EdgeTPU
echo "deb https://packages.cloud.google.com/apt coral-edgetpu-stable main" | sudo tee /etc/apt/sources.list.d/coral-edgetpu.list
curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
sudo apt update
#normal version
sudo apt install libedgetpu1-std
#Or the maximum clock version
# sudo apt install libedgetpu1-max
TF Lite installation
sudo pip3 install https://dl.google.com/coral/python/tflite_runtime-2.1.0.post1-cp37-cp37m-linux_armv7l.whl
Edge_TPU_Python_API installation
sudo apt install python3-edgetpu
【6. Jupyter Lab】 Install Jupyter Lab, which will give you a powerful Notebook environment.
6-1. Basic installation
[Installation] Install with the following command.
JupyterLab installation
sudo pip3 install jupyterlab
【Setting】 After the installation is completed, create a configuration file with the following command, and then edit the created configuration file. In the procedure below, you can connect from any terminal and set to require a password when connecting.
Creating a configuration file
jupyter notebook --generate-config
[Edit configuration file]
Edit configuration file
vi ~/.jupyter/jupyter_notebook_config.py
[Settings]
jupyter_notebook_config.py
c.NotebookApp.ip = '*'
c.NotebookApp.open_browser = False
c.NotebookApp.password = 'sha1:<Hashed password>'
You can get the hash of the password with the following command.
Password hashing
python3 -c 'from notebook.auth import passwd;print(passwd())'
(After executing the above, enter the characters you want to hash)
6-2. Launching Jputer Lab and turning it into a service
Start Jupyter Lab with the following command.
Start Jupyter Lab
jupyter lab
For service conversion, first prepare the configuration file for the service.
File preparation for service
sudo vi /etc/systemd/system/jupyter.service
The contents of the file are as follows.
/etc/systemd/system/jupyter.service
[Unit]
Desctiption = Jupyter Lab
After = syslog.target
[Service]
Type = simple
WorkingDirectory = /home/pi
Restart = always
ExecStart = /usr/local/bin/jupyter lab
User = pi
Group = pi
[Install]
WantedBy = multi-user.target
When the file is ready, enable and start the service with the following command.
command
#Service activation
sudo systemctl enable jupyter.service
#Start of the service
sudo systemctl start jupyter.service
6-3. Connect from a browser
With Jupyter Lab running above, connect to the following URL from your browser. In the above procedure, it is set to connect from other PCs, so please try connecting to Jupyter Lab from your own PC.
http://<ip_address>:8888/lab
You will be asked for a password, so enter the password you set above.
【7. Node-RED】
7-1. Installing Node-RED
You can create an environment to execute Node-RED by executing the following command.
Node-RED installation
bash <(curl -sL https://raw.githubusercontent.com/node-red/linux-installers/master/deb/update-nodejs-and-nodered)
The following message will be displayed. Press y and press Enter.
** If the old node.js is installed, it will be replaced with the new one, so please do not perform this procedure if there is a problem. ** **
This procedure assumes installation from a clean state, so execute it as it is.
This script will remove versions of Node.js prior to version 7.x, and Node-RED and
if necessary replace them with Node.js 10.x LTS (dubnium) and the latest Node-RED from Npm.
It also moves any Node-RED nodes that are globally installed into your user
~/.node-red/node_modules directory, and adds them to your package.json, so that
you can manage them with the palette manager.
It also tries to run 'npm rebuild' to refresh any extra nodes you have installed
that may have a native binary component. While this normally works ok, you need
to check that it succeeds for your combination of installed nodes.
To do all this it runs commands as root - please satisfy yourself that this will
not damage your Pi, or otherwise compromise your configuration.
If in doubt please backup your SD card first.
Are you really sure you want to do this ? [y/N] ?
The following message will be displayed, so select y to proceed.
Would you like to install the Pi-specific nodes ? [y/N] ?
(Do you want to install Pi-specific nodes?)
As the installation progresses, the following will be displayed.
Running Node-RED update for user pi at /home/pi on raspbian
This can take 20-30 minutes on the slower Pi versions - please wait.
Stop Node-RED ?
Remove old version of Node-RED ?
Remove old version of Node.js ?
Install Node.js LTS ? Node v12.16.0 Npm 6.13.7
Clean npm cache ?
Install Node-RED core ? 1.0.3
Move global nodes to local -
Install extra Pi nodes -
Npm rebuild existing nodes -
Add shortcut commands ?
Update systemd script ?
Any errors will be logged to /var/log/nodered-install.log
All done.
You can now start Node-RED with the command node-red-start
or using the icon under Menu / Programming / Node-RED
Then point your browser to localhost:1880 or http://{your_pi_ip-address}:1880
Started Wednesday, February 19, 2020 15:45:42 JST -Finished Wednesday, February 19, 2020 15:48:04 JST
7-2. Start Node-RED and make it a service
Start Node-RED with the following command.
Node-Launch RED
node-red
Use the following command to make Node-RED a service. By making it a service, Node-RED will be automatically executed when Raspberry Pi starts.
Node-RED service activation
sudo systemctl enable nodered.service
Service start(Automatic start only for the first time, after the next time)
sudo systemctl start nodered.service
7-3. Connect from a browser
With Node-RED running above, connect to the following URL from your browser.
http://<ip_address>:1880/
【Finally】
Although there are some issues, we were able to build the environment by following the procedure of the past. We have not checked the operation in detail yet, so please comment if you have any.
[Known bug]
I get an error when running ʻimport tensorflow` in python3
When I try to use TensorFlow and execute ʻimport tensorflow`, the following error is displayed.
2020-02-19 18:45:02.436414: E tensorflow/core/platform/hadoop/hadoop_file_system.cc:132] HadoopFileSystem load error: libhdfs.so: cannot open shared object file: No such file or directory
It has no effect so far, so I'm ignoring it at this time.
The version of TensorFlow that can be pip installed is old (as of 02/19/2020)
The latest version of TensorFlow that can be package-installed on Raspberry Pi is a little older, 1.14.0.
For tf1 series, I would like to use the latest version 1.15.0.
If you want to use the new version, you have to build it yourself.
Official procedure of TensorFlow says that it can be built in about 30 minutes, but I tried several times, but it took more than 3 hours. And finally fail ...
As for the version of TensorFlow, @PINTO has published a pre-built package, which makes me very happy.
At the moment, 1.15.0 is released for tf1 series, and 2.1.0 is released for tf2 series.
https://github.com/PINTO0309/Tensorflow-bin https://github.com/PINTO0309/Tensorflow-bin/#usage
In addition, if you use this package, the above error at the time of import will be solved.
Recommended Posts