[PYTHON] Periodically log the value of Omron environment sensor with Raspberry Pi
What is an environmental sensor?
Omron's environment sensors include temperature, humidity, illuminance, barometric pressure, noise, etc.
A device that combines multiple sensors together
The full range of functions that are typical of sensor manufacturers is attractive!
(Left: BAG type 2JCIE-BL01, Right: USB type 2JCIE-BU01)
In this article, we will get the value of the environmental sensor from Raspberry Pi. I uploaded it to a Google spreadsheet.
Try it out, from sensing to logging ** Can the functions up to this point be realized with only inexpensive general-purpose equipment? ** ** I am surprised. (Instead, there are many addictive points that make my heart break, but lol)
Things necessary
** ・ RaspberryPi ** (Pi3Model B is used this time) **-Python execution environment ** (This time, pyenv uses Python 3.7.6) ** ・ Google account ** (required to use spreadsheet) ** ・ Smartphone ** (for BLE Scanner setting) ** ・ Omron environment sensor (BAG type) ** * USB type will be created separately
procedure
** ① Check the Bluetooth connection between the Raspberry Pi and the sensor ** ** ② Change the environmental sensor mode to Broadcaster ** ** ③ Get the measured value of the environmental sensor with Python ** ** ④ Hit the GAS API from Python to write data to the spreadsheet ** ** ⑤ Script execution regularly **
Basically, Raspberry Pi is the starting point (central) of processing.
- Articles that I referred to ・ Data acquisition from the sensor https://ambidata.io/samples/temphumid/ble-gw-omron/ https://homemadegarbage.com/gobernables ・ Upload to Google Spreadsheet https://akerun.hateblo.jp/entry/2018/12/07/094714
Method
① Check the Bluetooth connection between the Raspberry pi and the sensor
Confirmation of recognition of environmental sensor
**-Check the operation of the environmental sensor ** Insert the batteries and make sure the lamp lights up for a moment
**-Scan for Bluetooth devices ** Execute the following command on Raspberry Pi
sudo hcitool lescan
LE Scan ...
DD:CC:BB:AA:77:66 Env
If you see the name "Env", this is the MAC address of the environmental sensor. If it does not come out, check the battery contact and the Bluetooth enable of Raspberry Pi.
Recognition confirmation with bluepy
bluepy is a library for accessing Bluetooth Low Energy (BLE) in Python (class definition)
Here, we will check if bluepy can recognize the environmental sensor.
** ・ Installation of required packages ** Install the following
sudo apt install libglib2.0-dev
** ・ Install bluepy **
Install with pip with the following command
pip install bluepy
** ・ Grant permissions to bluepy ** You need to give bluepy Sudo privileges for scanning.
Go to the folder where bluepy is installed and
cd ~.pyenv/versions/3.7.6/lib/python3.7/site-packages/bluepy
- The above is when Python 3.7.6 is installed with pyenv. Please note that the location varies depending on the environment
Grant Sudo privileges to bluepy-helper with the following command
sudo setcap 'cap_net_raw,cap_net_admin+eip' bluepy-helper
**-Creating a scanning code ** Create the code below
ble_scan.py
from bluepy import btle
class ScanDelegate(btle.DefaultDelegate):
def __init__(self): #constructor
btle.DefaultDelegate.__init__(self)
def handleDiscovery(self, dev, isNewDev, isNewData): #Scan handler
if isNewDev: #When a new device is found
print('found dev (%s)' % dev.addr)
scanner = btle.Scanner().withDelegate(ScanDelegate())
while True:
scanner.scan(10.0)
The above code works, What is important is ** Scanner class ** ** DefaultDelegate class ** ** ScanEntry class ** is.
In the above code **-Scanner.scan method: ** Set the argument (10 seconds in the above case) as the timeout value and execute the delegate specified by the Scanner.withDelegate method. **-DefaultDelegate class: ** Inheritance class for creating a delegate for the argument of Scanner.withDelegate. Describe specific processing in handleDiscovery method **-ScanEntry class: ** Holds device information acquired by Scanner.scan. The argument "dev" of handleDiscovery is equivalent Using,
** "Set 10 seconds as the timeout value and keep displaying the Mac address of the newly found device" **.
** ・ Execution of scanning code **
python ble_scan.py
As shown below, if the same MAC address as in LE Scan comes out, it is successful. (Note that the MAC address is lowercase)
found dev (dd:cc:bb:aa:77:66)
With the above, we have confirmed the environmental sensor recognition in bluepy.
② Change the mode of the environment sensor to Broadcaster
As described in this article, For data transmission (advertisement) of BLE devices ** ・ Connect mode: ** Get data by two-way communication **-Broadcaster: ** One-sided data transmission from device to central There are two types, ** Broadcaster is advantageous in terms of power consumption **. (Conversely, connect mode allows complicated operations to change device settings)
To increase the operating time of battery-powered environmental sensors Change to Broadcaster and proceed with the following operations
Install BLE Scanner
A tool called BLE Scannear used to change modes, Please install it on your smartphone from the following Android iOS
Change the advertisement setting of the environmental sensor
Change the advertisement settings and set the mode to Broadcaster. You can also change the advertisement transmission interval here (for applications that require speed)
**-Open BLE Scanner and CONNECT to the device labeled "Env" **
** ・ "0C4C3042 ~" is the advertisement setting, so look for it **
** ・ How to read the advertisement settings in (2) above **
If you read the OMRON Official User's Manual, it will look like the figure below.
** ・ If you want to change to broadcast mode: Set Beacon Mode to 02 or 04 **
** ・ If you want to shorten the update interval: Reduce the advertisement transmission interval **
It is necessary to change.
- What is "Limited Broadcaster"? You might think, but it's probably a setting that advertises intermittently to save battery power. I will not use it this time because it describes the risk of becoming unrecognizable.
Beacon Mode is wondering whether it is 02 or 04, but This time, we will focus on the environmental index Before：0808A0000A003200 08 00 After：0808A0000A003200 04 00 will do. (If you use an accelerometer, set it to 02)
** ・ Change advertisement settings **
Click ③ "W" in the above figure, change it as shown in the figure below, and press OK.
** ・ Confirmation of change **
If you go back to the top and the device name changes to "EP", you are successful.
Shorten the update interval of the environmental sensor
The sensor value update interval is 300 seconds (5 minutes) by default. This is too long! I think that there are many people, so I will change it short.
** ・ "0C4C3011 ~" is the update interval setting, so look for it **
Following the previous operation, connect to the BLE Scanner device "EP" and
Look for "0C4C3011 ~"
** ・ How to read the update interval setting in (2) above **
It is the update interval in seconds converted to hexadecimal.
However, because it is a beginner crying specification that the upper byte and the lower byte are reversed **
be careful.
This time, enter "0A00" to change the interval to 10 seconds as shown in the figure below.
** ・ Confirmation of change of update interval **
If you press "R" and the contents of Hex are updated, it is successful.
If you click "RAW DATA" on the top screen, you can see the measured values of the sensor.
The specifications of the measured values are as shown in the figure below.
- Please note that ** bytes are upside down as in the example **
③ Get the measured value of the environmental sensor with Python
Since it was found that the measured value of the sensor can be obtained in (2), Increase the versatility of processing with Python.
Create a class for acquiring sensor measurements
Create a class to get sensor measurements with bluepy. In order to pass the processing at the time of scanning as a delegate as in ① and time, Create code like the one below.
The acquired sensor measurement value is stored as a dict for each sensor type in the variable "sensorValue".
omron_env_broadcast.py
from bluepy import btle
import time
import struct
class ScanDelegate(btle.DefaultDelegate):
#constructor
def __init__(self):
btle.DefaultDelegate.__init__(self)
#Variables for holding sensor data
self.sensorValue = None
#Scan handler
def handleDiscovery(self, dev, isNewDev, isNewData):
#When a new device is found
if isNewDev or isNewData:
#Extract advertisement data
for (adtype, desc, value) in dev.getScanData():
#Data retrieval is executed when the environment sensor is used.
if desc == 'Manufacturer' and value[0:4] == 'd502':
#Take out the sensor type (EP or IM)
sensorType = dev.scanData[dev.SHORT_LOCAL_NAME].decode(encoding='utf-8')
#Extraction of sensor data for EP
if sensorType == 'EP':
self.decodeSensorData_EP(value)
#Extraction of sensor data during IM
if sensorType == 'IM':
self.decodeSensorData_IM(value)
#Take out sensor data and convert it to dict format (in EP mode)
def decodeSensorData_EP(self, valueStr):
#Sensor data from character strings(6th and subsequent characters)Only take out and convert to binary
valueBinary = bytes.fromhex(valueStr[6:])
#Convert binary sensor data to integer Tuple
(temp, humid, light, uv, press, noise, discomf, wbgt, rfu, batt) = struct.unpack('<hhhhhhhhhB', valueBinary)
#Store in dict type after unit conversion
self.sensorValue = {
'SensorType': 'EP',
'Temperature': temp / 100,
'Humidity': humid / 100,
'Light': light,
'UV': uv / 100,
'Pressure': press / 10,
'Noise': noise / 100,
'Discomfort': discomf / 100,
'WBGT': wbgt / 100,
'BatteryVoltage': (batt + 100) / 100
}
#Take out sensor data and convert it to dict format (in IM mode)
def decodeSensorData_IM(self, valueStr):
#Sensor data from character strings(6th and subsequent characters)Only take out and convert to binary
valueBinary = bytes.fromhex(valueStr[6:])
#Convert binary sensor data to integer Tuple
(temp, humid, light, uv, press, noise, accelX, accelY, accelZ, batt) = struct.unpack('<hhhhhhhhhB', valueBinary)
#Store in dict type after unit conversion
self.sensorValue = {
'SensorType': 'IM',
'Temperature': temp / 100,
'Humidity': humid / 100,
'Light': light,
'UV': uv / 100,
'Pressure': press / 10,
'Noise': noise / 100,
'AccelerationX': accelX / 10,
'AccelerationY': accelY / 10,
'AccelerationZ': accelZ / 10,
'BatteryVoltage': (batt + 100) / 100
}
In the following part, the sensor measurement value is acquired from the advertisement data.
#Data retrieval is executed when the environment sensor is used.
if desc == 'Manufacturer' and value[0:4] == 'd502':
#Take out the sensor type (EP or IM)
sensorType = dev.scanData[dev.SHORT_LOCAL_NAME].decode(encoding='utf-8')
#Extraction of sensor data for EP
if sensorType == 'EP':
self.decodeSensorData_EP(value)
#Extraction of sensor data during IM
if sensorType == 'IM':
self.decodeSensorData_IM(value)
The above process was created based on the following. ・ Environmental sensor can be identified by "desc ='Manufacturer' & value 0-4 characters is'd502'" -Sensor modes and measured values are held in the form shown in the following [Reference]. -Since the acquired measured value is binary data and the format differs depending on whether the mode is EP or IM, create a method "decodeSensorData" for each mode to convert it to the numerical Dict format for ease of use.
** [Reference] Results of debugging advertisement data in BroadCast mode with VS Code **
Creating main code
Create the main code to execute the above sensor value acquisition class. As in (2), set the sensor value acquisition class in the Scanner delegate and set it. Scan and run.
omron_env_toSpreadSheet.py
from bluepy import btle
from omron_env_broadcast import ScanDelegate
#omron_env_broadcast.Set the sensor value acquisition delegate of py to execute at scan time
scanner = btle.Scanner().withDelegate(ScanDelegate())
#Scan to get sensor value (timeout 5 seconds)
scanner.scan(5.0)
#Display the temperature as a trial
print(scanner.delegate.sensorValue['Temperature'])
#Describe the process of uploading to Google Spreadsheet in ④
Try running from the console
python omron_env_toSpreadSheet.py
25.98
You have now obtained the sensor readings in Python. The acquired measurements are stored in scanner.delegate.sensorValue.
④ Hit the GAS API from Python to write data to the spreadsheet
Output the acquired data to a Google spreadsheet.
** * Most people want to output other than spreadsheets, so ** ** In that case, skip this chapter and move to ⑤ ** (There are many options such as CSV output with Pandas and visualization with Ambient)
Create GAS script
Perform this work on your PC.
Go to Google Spreadsheets and go to
Create a spreadsheet like the one below
Note that the sheet name should be the same as the device name **
Select "Tools"-> "Script Editor" and select Create a GAS script like the one below
postSensorData.gs
var spreadsheetId = '******'//← Enter the spreadsheet ID
//Receive the posted data
function doPost(e){
var data = [
e.parameter.Date_Master, //Master date and time
e.parameter.Date, //Measurement date and time
e.parameter.Temperature, //temperature
e.parameter.Humidity, //Humidity
e.parameter.Light, //Illuminance
e.parameter.UV, // UV
e.parameter.Pressure, //pressure
e.parameter.Noise, //noise
e.parameter.BatteryVoltage, //Battery voltage
new Date(), //Upload end time
];
//Describe the acquired data in the log
Logger.log(new Date());
//Write a row of data to a spreadsheet
addData(e.parameter.DeviceName, data);
return ContentService.createTextOutput("ok");
}
//Write a row of data to a spreadsheet
function addData(sheetName, data){
var sheet = SpreadsheetApp.openById(spreadsheetId).getSheetByName(sheetName);
sheet.appendRow(data);
}
** * Apparently, the Function name must be "doPost" to post using the API **
- The URL of the spreadsheet ID is
https://docs.google.com/spreadsheets/d/AAAAAAA/edit
If so, the "AAAAAAA" part corresponds
Publish the script as a Web API
"Publish" → "Introduced as a web application"
The following screen will appear, so
・ Project version → Version you want to make into API (At first, select "New" to OK)
・ Execute the app as → Me (self)
・ Who has access to the app → Anyone, even anonimous (all (including anonymous users))
Select to update (Deploy)
- If Google authentication comes out, follow the instructions to authenticate.
If "This app has not been confirmed" appears, you can clear it by following the link from "Details" as shown in the figure below.
If successful, the following screen will appear.
Make a note of the API URL as you will use it later.
API external execution test
** * From here, the work will be done on the Raspberry Pi side ** Hit the API with curl to send the appropriate data and test if it can be POSTed from the outside. Execute the following command (install curl if it is not included)
curl -L [API URL] -F 'sheetName=[Sheet name]' -F "Date_Master=4.1" -F "Date=4.2" -F "SensorType=4.2" -F "Temperature=4.4" -F "Humidity=4.4" -F "Light=4.1" -F "UV=4.2" -F "Pressure=4.4" -F "Noise=4.4" -F "BatteryVoltage=4.4" -F "DeviceName=[Sheet name]"
Success if the values are entered in the spreadsheet
- When things go wrong I also failed several times, but most of the time it was caused by the following GAS script. -The function name of the script is not "doPost" -The "doPost" function was not selected when the script API was published. -The latest version (New) was not selected when the script API was published.
Install Requests
Install the Requests library to perform POST on the Python side
pip install requests
Added upload process to Python code
In the code "omron_env_toSpreadSheet.py" created in ③, Add the process of uploading data by hitting the API of the GAS script.
omron_env_toSpreadSheet.py
from bluepy import btle
from omron_env_broadcast import ScanDelegate
from datetime import datetime, timedelta
import requests
#Get the current time
date = datetime.today()
#Round the current time in minutes
masterDate = date.replace(second=0, microsecond=0)
if date.second >= 30:
masterDate += timedelta(minutes=1)
#omron_env_broadcast.Set the sensor value acquisition delegate of py to execute at scan time
scanner = btle.Scanner().withDelegate(ScanDelegate())
#Scan to get sensor value
scanner.scan(5.0)
######Processing to upload to Google Spreadsheet######
#Upload only when the sensor value is not None
if scanner.delegate.sensorValue is not None:
#Device name
deviceName = '****'#← Make it the same name as the sheet name of the spreadsheet
#Data to POST
data = {
'DeviceName': deviceName,
'Date_Master': str(masterDate),
'Date': str(date),
'SensorType': str(scanner.delegate.sensorValue['SensorType']),
'Temperature': str(scanner.delegate.sensorValue['Temperature']),
'Humidity': str(scanner.delegate.sensorValue['Humidity']),
'Light': str(scanner.delegate.sensorValue['Light']),
'UV': str(scanner.delegate.sensorValue['UV']),
'Pressure': str(scanner.delegate.sensorValue['Pressure']),
'Noise': str(scanner.delegate.sensorValue['Noise']),
'BatteryVoltage': str(scanner.delegate.sensorValue['BatteryVoltage'])
}
#API URL
url = 'https://script.google.com/macros/s/******/exec'#← Enter the API URL
#POST data to API
response = requests.post(url, data=data)
- Since the Date_Master column (masterDate variable) was created for the join key when there are multiple sensors, Don't worry about it at this time.
Run the Python script to check the upload
Execute the above Python script with the following command
python omron_env_toSpreadSheet.py
As shown in the figure below, if the sensor value is entered in the spreadsheet, it is successful.
⑤ Periodic execution of script
With the above method, it is troublesome to execute the script every time, so Automate using the periodic package "cron". It looks like a Windows task scheduler.
Enable cron
It may be disabled by default, so enable it by referring to here.
**-Checkpoint 1: Check rsyslog.conf ** It will not work if "cron" in /etc/rsyslog.conf is commented out. In my case, it was commented out as follows, so
###############
#### RULES ####
###############
#
# First some standard log files. Log by facility.
#
auth,authpriv.* /var/log/auth.log
*.*;auth,authpriv.none -/var/log/syslog
#cron.* /var/log/cron.log
daemon.* -/var/log/daemon.log
After uncommenting as below,
cron.* /var/log/cron.log
Restart rsyslog with the following command.
sudo /etc/init.d/rsyslog restart
** ・ Checkpoint 2: Change log level ** In / etc / default / cron, specify the items to be described in the log when cron is executed. By default, it seems that the log is not output as shown below, so
# For quick reference, the currently available log levels are:
# 0 no logging (errors are logged regardless)
# 1 log start of jobs
# 2 log end of jobs
# 4 log jobs with exit status != 0
# 8 log the process identifier of child process (in all logs)
#
#EXTRA_OPTS=""
Uncomment EXTRA_OPTS as shown below
EXTRA_OPTS='-L 15'
will do. It means that all will be output with 1 + 2 + 4 + 8 = 15.
Restart cron with the following command.
sudo /etc/init.d/cron restart
If cron.log is generated in / var / log, it is successful. (Please check here if you want to see the execution history of cron)
Schedule execution with cron
Register periodic execution with cron
** ・ Editing crontab ** Open crontab with the following command
crontab -e
- "Sudo crontab -e" will open a different file, so please do not add sudo.
When asked which editor to open, select the editor you like (nano recommended for beginners)
# Edit this file to introduce tasks to be run by cron.
#
# Each task to run has to be defined through a single line
# indicating with different fields when the task will be run
: Various things continue
There are various comments written as above, At the very end, describe the timing and command you want to execute. The timing format is [here](http://tech.junkpot.net/archives/721/crontab-%E3%81%A7%E7%B4%B0%E3%81%8B%E3%81%84%E3 % 82% B9% E3% 82% B1% E3% 82% B8% E3% 83% A5% E3% 83% BC% E3% 83% AB% E3% 81% AE% E8% A8% AD% E5% AE Please refer to% 9A.html)
This time, since it is executed every 5 minutes, the following contents are described.
*/5 * * * * [Python full path] [omron_env_toSpreadSheet.full path of py] >/dev/null 2>&1
- Python requires full path specification as shown in here. Below is an example of a full path -Default Python3: / usr / bin / python3 -Pyenv environment: /home/[username]/.pyenv/shims/python
- The last "> / dev / null 2> & 1" is a process to prevent sending an email. [Here](https://orebibou.com/2014/12/cron%E3%81%A7%E3%82%B9%E3%82%AF%E3%83%AA%E3%83%97%E3% 83% 88% E3% 82% 92% E5% AE% 9A% E6% 9C% 9F% E5% AE% 9F% E8% A1% 8C% E3% 81% 95% E3% 81% 9B% E3% 82% See 8B% E3% 81% A8% E3% 81% 8D% E3% 81% AB% E6% B3% A8% E6% 84% 8F% E3% 81% 97% E3% 81% 9F /)
** ・ Start cron ** Start cron with the following command
sudo /etc/init.d/cron start
Wait for a while above and you're done if the spreadsheet is updated every 5 minutes!
- If it doesn't work, see the cron log in / var / log / syslog and deal with it. ** Thank you for your hard work! ** **
in conclusion
Increase the number of sensors and realize your dream IoT house! ** It looks like you can do it. Omron's sensors are multifunctional and of high quality, but they are not cheap enough to buy pompoms, so I will try even a cheaper sensor!
Recommended Posts