Raspberry Pi + python + IoT device, environment construction procedure to start image processing and machine learning

Introduction

In order to start image processing and machine learning using cameras and detection devices with Raspberry Pi and python, I will summarize how to build a stable environment through trial and error.

Since my Raspberry Pi is Pi2B, I use a USB-connected wi-fi module.

Basic environment

Prepare the basic environment on the premise that a wired LAN connection is possible.

Localizing into Japanese

sudo dpkg-reconfigure locales

Select the following in the CUI and ** OK **.

sudo apt-get install ttf-kochi-gothic xfonts-intl-japanese xfonts-intl-japanese-big xfonts-kaname
sudo apt-get install uim uim-mozc

ClamAV Antivirus software ClamAV that also supports raspbian. You need to run it with cron to make the scan automatic.

sudo apt-get install clamav clamdscan

editor

I like emacs, so install it.

sudo apt-get install emacs

Add user, add group

** pi ** Users are concerned about security, so create a new user for development. Enter your username in ** NAME **.

sudo adduser NAME
sudo gpasswd -a NAME sudo
sudo usermod -aG audio,video,plugdev,input,netdev,spi,i2c,gpio,dialout,cdrom NAME

Partition extension

You can do it with raspi-config, but this is a reference when you do it with fdisk. Eliminates the narrow effective area even when using a large-capacity SD card.

df -kh
sudo fdisk /dev/mmcblk0
p
d
2
p
n
p
2
w
sudo reboot
sudo resize2fs /dev/mmcblk0p2

Specify the start address of the second partition displayed by the first p, and do not enter anything. (OK by default)

NTP Time adjustment.

sudo apt-get install ntp
sudo emacs /etc/ntp.conf

Edit /etc/ntp.conf and change to Japanese NTP server.

/etc/ntp.conf


...
...
#server 0.debian.pool.ntp.org iburst   //Comment
#server 1.debian.pool.ntp.org iburst   //Comment
#server 2.debian.pool.ntp.org iburst   //Comment
#server 3.debian.pool.ntp.org iburst   //Comment
server ntp1.jst.mfeed.ad.jp
server ntp2.jst.mfeed.ad.jp
server ntp3.jst.mfeed.ad.jp
...
...
sudo service ntp restart
ntpq -p

rpi-clone SD card backup command rpi-clone. It is possible to clone to SD cards with different capacities!

git clone https://github.com/billw2/rpi-clone.git
cd rpi-clone

The best procedure is to look at the readme, but it's a memorandum. Insert the card reader into the USB port, insert the SD card, and check the device name with fdisk.

sudo fdisk -l

Execute the clone command, answer the settings interactively and execute it, and after a while it is completed.

sudo ./rpi-clone sda -f

If you create a backup script like the one below, you can manage when you cloned and what you typed in the log text.

mkclone.scr


#!/bin/sh
dnow=`date +%Y%m%d-%H%M%S`

sudo ./rpi-clone sda -f | tee mkclone_${dnow}.log
echo '## Written install log to "mkclone_${dnow}.log"'

network

wi-fi For ** SSID ** and ** PWD **, enter the one at home to generate encrypted information.

sudo sh -c 'wpa_passphrase SSID PWD >> /etc/wpa_supplicant/wpa_supplicant.conf'
cd /etc/wpa_supplicant/
sudo cp -p wpa_supplicant.conf wpa_supplicant.conf.ORG
sudo emacs wpa_supplicant.conf

Edit /etc/wpa_supplicant/wpa_supplicant.conf.

/etc/wpa_supplicant/wpa_supplicant.conf


ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
update_config=1
network={
        ssid="SSID"
        psk=???????????????????????????????????????????????????????????????
}

You can use it by restarting.

sudo /etc/init.d/networking restart

Fixed IP

Settings when you don't like DHCP. The IP address is an example, and it is fixed to ** 192.168.1.3 **.

cd /etc/network/
sudo cp -p interfaces interfaces.ORG
sudo emacs interfaces

Edit / etc / network / interfaces.

/etc/network/interfaces


source-directory /etc/network/interfaces.d

auto lo
iface lo inet loopback

iface eth0 inet manual

allow-hotplug wlan0
#iface wlan0 inet manual
iface wlan0 inet static
address 192.168.1.3
netmask 255.255.255.0
network 192.168.1.0
broadcast 192.168.1.255
gateway 192.168.1.1
dns-nameservers   192.168.1.1
    wpa-conf /etc/wpa_supplicant/wpa_supplicant.conf

allow-hotplug wlan1
iface wlan1 inet manual
    wpa-conf /etc/wpa_supplicant/wpa_supplicant.conf

** dns-nameservers 192.168.1.1 <= Without this you will not be able to access externally **

If you restart, it will become a fixed IP. Check with ʻifconfig`.

sudo /etc/init.d/networking restart

http For the time being, finish up to the LAMP server.

sudo apt-get install apache2 apache2-utils

mysql For the time being, finish up to the LAMP server.

sudo apt-get install mysql-server libmysqlclient-dev

PHP For the time being, finish up to the LAMP server.

sudo apt-get install php5 phpmyadmin

samba Since Raspberry Pi is basically accessed remotely, make it possible to read and write files from windows.

sudo apt-get install samba
sudo emacs /etc/samba/smb.conf

Edit /etc/samba/smb.conf.

/etc/samba/smb.conf


...
...
 [global]workgroup = YourWorkGrp 
...
...
 [homes]
browseable = no
read only = no
create mask = 0640
directory mask = 0750
...
...

Enter the pi user or newly created user in ** NAME **.

sudo smbpasswd -a NAME
sudo /etc/init.d/samba start

ftp Basically samba and rcp, but also include ftp. Enter the pi user or newly created user in ** NAME **.

sudo apt-get install pure-ftpd
sudo groupadd ftpgroup
sudo mkdir /home/NAME/FTP
sudo useradd ftpuser -g ftpgroup -s /sbin/nologin -d /dev/null
sudo chown -R ftpuser:ftpgroup /home/NAME/FTP
sudo pure-pw useradd upload -u ftpuser -g ftpgroup -d /home/NAME/FTP -m
sudo ln -s /etc/pure-ftpd/conf/PureDB /etc/pure-ftpd/auth/60puredb

VNC I don't use it much because it will be heavy, but I will put it in for the time being.

sudo apt-get upgrade realvnc-vnc-server realvnc-vnc-viewer

start VNC service sudo systemctl start vncserver-x11-serviced.service sudo systemctl enable vncserver-x11-serviced.service

stop VNC service sudo systemctl stop vncserver-x11-serviced.service sudo systemctl disable vncserver-x11-serviced.service

Development environment (hardware)

IoT devices are used for I2C and SPI connections.

I2C If you can't do this, you can't talk.

sudo apt-get install i2c-tools python-smbus

fritzing God software fritzing for creating connection diagrams. I don't use it on Raspberry Pi, but I confirmed that it can be installed.

sudo apt-get inatall fritzing

Development environment (software system)

The development language basically uses python3.

subversion Familiar configuration management software. Maybe git nowadays. .. ..

sudo apt-get install subversion

doxygen Documentation software.

sudo apt-get install doxygen

python Python will be the main development language.

sudo apt-get install python-pip python-dev
sudo apt-get install python-scipy
sudo apt-get install python-matplotlib
sudo apt-get install libopencv-dev python-opencv
sudo apt-get install python-opengl

I will delete it later, but check the version of opencv to check the environment.

python -c "import cv2.cv as cv; print(cv.__version__)"

virtualenv Switch environment of python. I've included it, but I haven't used it yet, so typing python by default results in python2. .. ..

sudo pip install virtualenv virtualenvwrapper
sudo rm -rf ~/.cache/pip

Edit ~ / .profile.

emacs ~/.profile

~/.profile


...
...
# virtualenv and virtualenvwrapper
export WORKON_HOME=$HOME/.virtualenvs
source /usr/local/bin/virtualenvwrapper.sh

Manipulate .profile to reflect.

source ~/.profile

make "cv" workspace

mkvirtualenv cv -p python3
source ~/.profile
workon cv

picamera Required when using picamera.

sudo apt-get install python-picamera python3-picamera
sudo apt-get install python-picamera-docs

twitter Twitter is one of the leading candidates for IoT output destinations. Put the library so that you can throw information from python. There is a lot, but I chose the first twython.

sudo pip install twython
sudo pip3 install twython
sudo pip install feedparser
sudo pip3 install feedparser

cffi Interface library cffi for calling C functions from python. You can do it with ctypes, but here's a clean description.

sudo apt-get install libffi-dev
sudo pip install cffi
sudo pip3 install cffi
sudo pip3 install smbus

requests python's HTTP library [requests] link-01. Even when accessing the cloud, I want to write code in a general-purpose library as much as possible.

pip3 install requests
pip install requests

java I think I'll use it for something, so just in case.

sudo apt-get install oracle-java7-jdk

Image processing

opencv3 The latest opencv is compiled and installed by yourself. Currently it worked up to 3.1.0, but 3.2.0 was NG.

sudo apt-get update
sudo apt-get upgrade

Remove default (old) OpenCV

sudo apt-get remove libopencv*
sudo apt-get autoremove

Install OpenCV 3.1.0

sudo apt-get install build-essential cmake cmake-curses-gui pkg-config
sudo apt-get install libjpeg-dev libtiff5-dev libjasper-dev libpng12-dev libavcodec-dev libavformat-dev libswscale-dev libeigen3-dev libxvidcore-dev libx264-dev libgtk2.0-dev
sudo apt-get -y install libv4l-dev v4l-utils
sudo apt-get install libatlas-base-dev gfortran
sudo apt-get install python3-dev python3-numpy

cd ~/download
wget -O opencv-3.1.0.zip https://github.com/Itseez/opencv/archive/3.1.0.zip
wget -O opencv-3.1.0_contrib.zip https://github.com/opencv/opencv_contrib/archive/3.1.0.zip
unzip opencv-3.1.0.zip
unzip opencv-3.1.0_contrib.zip

cd opencv-3.1.0
mkdir build
cd build

cmake -D CMAKE_BUILD_TYPE=RELEASE \
      -D CMAKE_INSTALL_PREFIX=/usr/local \
      -D INSTALL_PYTHON_EXAMPLES=ON \
      -D OPENCV_EXTRA_MODULES_PATH=~/download/opencv_contrib-3.1.0/modules \
      -D BUILD_EXAMPLES=ON ..

make
sudo make install
sudo ldconfig

python3 -c "import cv2 as cv; print(cv.\_\_version\_\_)"

imagemagick Image processing, simple image viewer.

sudo apt-get install imagemagick

Check the operation with the following command. convert raspberrypi.png -resize 128x64 -gravity center -background black -extent 128x64 raspberrypi.ppm

movie tools An alternative to ffmpeg.

ffmpeg / avconv / mencoder all replaced. libav-tools and mpv are the updated packages in Jessie: https://www.raspberrypi.org/forums/viewtopic.php?f=66&t=123442

sudo apt-get install libav-tools
sudo apt-get install openshot

tesseract-ocr + pyocr OCR software tesseract-ocr that also supports Japanese. Select pyocr as the wrapper to call from python.

apt-get install tesseract-ocr libtesseract-dev libleptonica-dev
tesseract -v
wget https://sourceforge.net/projects/tesseract-ocr-alt/files/tesseract-ocr-3.02.jpn.tar.gz
tar xvfz tesseract-ocr-3.02.jpn.tar.gz
cd tesseract-ocr/tessdata/
sudo cp jpn.traineddata /usr/share/tesseract-ocr/tessdata/
sudo pip install pyocr
sudo pip3 install pyocr

When you enter the image file ** IMG ** as shown below, the characters are parsed and saved in a text file.

python


tesseract IMG out -l jpn
cat out.txt

Speech processing

As with images, audio is important.

espeak Speak command. Only English is supported. .. ..

sudo apt-get install espeak
sudo apt-get install python-espeak
sudo apt-get install python3-espeak

You can speak Japanese like this. .. ..

espeak -g 20 -w a.wav "anata ga ill"
espeak -g 20 -w w.wav "watashi ga ill"

OpenJtalk Can speak Japanese OpenJTalk. English words become alphabet reading. .. ..

sudo apt-get install open-jtalk open-jtalk-mecab-naist-jdic hts-voice-nitech-jp-atr503-m001
wget https://sourceforge.net/projects/mmdagent/files/MMDAgent_Example/MMDAgent_Example-1.6/MMDAgent_Example-1.6.zip/download -O MMDAgent_Example-1.6.zip
unzip MMDAgent_Example-1.6.zip
sudo cp -r MMDAgent_Example-1.6/Voice/mei/ /usr/share/hts-voice

mplayer It's not a development environment, but I want to develop while listening to Internet radio on Raspberry Pi.

sudo apt-get install mplayer
mplayer -playlist shoutcast_???.pls

Internet radio that can get playlists (.pls) is, for example, shoutcast.

Machine learning

Areas where you want to deepen your understanding by linking with IoT.

TensorFlow Check tensorflow-on-raspberry-pi for the latest version. This is for 1.0.0.

For Python 2.7

wget https://github.com/samjabrahams/tensorflow-on-raspberry-pi/releases/download/v1.0.0/tensorflow-1.0.0-cp27-none-linux_armv7l.whl
sudo pip install tensorflow-1.0.0-cp27-none-linux_armv7l.whl
sudo pip uninstall mock
sudo pip install mock
python -c "import tensorflow as tf; print(tf.__version__)"

For Python 3.3+

wget https://github.com/samjabrahams/tensorflow-on-raspberry-pi/releases/download/v1.0.0/tensorflow-1.0.0-cp34-cp34m-linux_armv7l.whl
sudo pip3 install tensorflow-1.0.0-cp34-cp34m-linux_armv7l.whl
sudo pip3 uninstall mock
sudo pip3 install mock
python3 -c "import tensorflow as tf; print(tf.__version__)"

keras If you use TensorFlow, keras makes it very easy to use.

sudo pip install keras
sudo pip3 install keras
sudo apt-get install python3-numpy
sudo apt-get install python3-scipy
sudo apt-get install python3-pandas
sudo apt-get install python3-h5py

In some cases, this one (I don't remember, but when I used pyenv, it was ubuntu instead of Raspberry Pi).

pip3 install numpy
pip3 install scipy
pip3 install pandas
pip3 install h5py
pip3 install pillow
pip install numpy
pip install scipy
pip install pandas
pip install h5py
pip install pillow

Mecab Morphological analysis engine Mecab. Make it available from python as well.

sudo apt-get install mecab libmecab-dev mecab-ipadic-utf8 python-mecab
sudo pip3 install mecab-python3
python3 -c "import MeCab; print(MeCab.__version__)"

Google Cloud SDK SDK of Google Cloud Platform that is evolving day by day ? hl = ja). However, in python, we aim to implement using [requests] link-01 without using this SDK.

export CLOUD_SDK_REPO="cloud-sdk-$(lsb_release -c -s)"
echo "deb https://packages.cloud.google.com/apt $CLOUD_SDK_REPO main" | sudo tee -a /etc/apt/sources.list.d/google-cloud-sdk.list
curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -

sudo apt-get apt-transport-https
sudo apt-get update
sudo apt-get install google-cloud-sdk

Recommended Posts

Raspberry Pi + python + IoT device, environment construction procedure to start image processing and machine learning
Python and machine learning environment construction (macOS)
Easy IoT to start with Raspberry Pi and MESH
From setting up Raspberry Pi to installing Python environment
Machine learning with Raspberry Pi 4 and Coral USB Accelerator
Preparing to start "Python machine learning programming" (for macOS)
Procedure to exe python file from Ubunts environment construction
Anaconda3 python environment construction procedure
How to use machine learning for work? 03_Python coding procedure
Python environment construction and TensorFlow
[Let's play with Python] Image processing to monochrome and dots
Machine learning environment construction macbook 2021
Python environment construction and SQL execution example to DB and memo of basic processing for statistics 2019
Translate I2C device driver for Arduino to Python for Raspberry pi
Connect Raspberry Pi to Alibaba Cloud IoT Platform with Python
Building a Raspberry Pi for studying Python and machine learning (RaspberryPi4 & Buster version (RaspberryPi3 is also possible))
Environment construction of python and opencv
Python + Unity Reinforcement learning environment construction
Image recognition environment construction and basics
[Raspberry Pi] Changed Python default to Python3
Handmade Alexa Festival! Serverless, Invitation to IoT + Voice. Raspberry Pi + Alexa Voice Servce (Python)
I connected the thermo sensor to the Raspberry Pi and measured the temperature (Python)
Stream processing for Python and SensorTag, Kafka, Spark Streaming --Part 1: Raspberry Pi 3
Python & Machine Learning Study Memo: Environment Preparation
Python (anaconda) development environment construction procedure (SpringToolsSuites) _2020.4
Python project environment construction procedure (for windows)
I installed Python 3.5.1 to study machine learning
[Python] Road to snake charmer (1) Environment construction
Environment construction procedure: Ubuntu + Apache2 + Python + Pyramid
An introduction to Python for machine learning
Build AI / machine learning environment with Python
Environment construction procedure to operate chrome without installing python on Windows (using selenium)
[Super Introduction] Machine learning using Python-From environment construction to implementation of simple perceptron-
Build a python environment to learn the theory and implementation of deep learning
Note: Start Anaconda, enter the Python virtual environment, and connect locally to MongoDB.
I tried to process and transform the image and expand the data for machine learning
Python learning notes for machine learning with Chainer Chapters 11 and 12 Introduction to Pandas Matplotlib
Use Raspberry Pi Python to TMP36 analog temperature sensor and MCP3008 AD converter
Python3 TensorFlow environment construction (Mac and pyenv virtualenv)
[Python] Easy introduction to machine learning with python (SVM)
VS Code + Azure Functions + Python environment construction procedure
Image characters and post to slack (python slackbot)
How to use Raspberry Pi pie camera Python
EV3 x Pyrhon Machine Learning Part 1 Environment Construction
Installation of Python3 and Flask [Environment construction summary]
Image Processing with Python Environment Setup for Windows
Easy connection between Raspberry Pi and AWS IoT
Python mock to try AWS IoT Device Shadow
[Django3] Environment construction and various settings summary [Python3]
Raspberry Pi and AWS IoT connection program example
From Python environment construction to virtual environment construction with anaconda
Connect to MySQL with Python on Raspberry Pi
Build a Python machine learning environment with a container
Build a Python development environment on Raspberry Pi
Script to easily create a client device environment for AWS IoT (Python v2 version)
Until you create a machine learning environment with Python on Windows 7 and run it
Distributed environment construction with Raspberry PI series (Part 4: NFS server construction and client OS import)
I tried to build an environment for machine learning with Python (Mac OS X)