[PYTHON] Pull out smart meter information with the cheapest hardware

Introduction

With the liberalization of electricity, the installation of ** smart meters ** is underway.

By introducing ** smart meters ** installed by electric power companies and ** HEMS visualization terminal-like products ** sold by each company, ** power consumption etc. can be viewed in real time **. I will.

However! , ** HEMS visualization terminal-like thing ** is ** housing remodeling industry price setting ** very ** expensive **.

Therefore, the purpose of this article is to make ** HEMS visualization terminal-like things ** at a low price **.

Notice

It is assumed that the story is in the TEPCO supply area, but if you live in another company's supply area, please read the name of the electric power company m (_ _) m

The physical layer of communication is decided by each company, but when I opened the lid, it was unified with Wi-SUN (described later) nationwide. The hardware you should buy is the same anywhere in the country.

hardware

Have a smart meter installed

First of all, you have to have a smart meter installed.

** Q: But it's expensive, isn't it? ** ** ** A: No, it's free! ** **

Originally, it will be replaced with a smart meter even if you leave it alone, but if you ask "Do it first", the replacement work will be given priority.

In addition, the liberalization of electric power has made it possible to select an electric power company, but if you change the electric power company, it will automatically be replaced with a smart meter, so it is recommended that you switch to another one.

Which power company you choose to save money depends on the amount of electricity used by each household, so if you compare it with Price.com etc. It will be good.

If the amount of electricity used is low, such as in a single-person household, it may be cheaper to leave TEPCO as it is.

Apply for the B route service

Suddenly, a word I didn't know came out.

The ** B route ** is the communication route between the smart meter and your home HEMS device. (Although routes A and C are also terms, explanations are omitted because they are side roads.)

When you apply for the B route service, you will be given your smart meter ID and password. The usage fee is free.

Even if you switch power companies due to the liberalization of electricity, the application destination for the B route service will be TEPCO. To be precise, it is ** TEPCO Power Grid, Inc. **, which is in charge of power transmission and distribution.

Even before the smart meter is replaced, if you apply for the B route service, you are implicitly applying for the smart meter.

Let's apply from the link below Electricity meter information transmission service (B route service) (Other than TEPCO, please try google with the power company name and B route service)

In the note on that page

  • Customers are requested to prepare the equipment in the customer's home such as HEMS equipment.

It is written that "equipment in the customer's home such as HEMS equipment" will be made at the bottom of this article.

By the way, the ID will be sent by mail and the password will be sent by email. Isn't it the other way around!

By the way, the envelope to which the ID is sent is a shoboy that the sender is stamped with a stamp. It makes me feel the lack of motivation of TEPCO ~~. DSC_0472.JPG

Buy a Wi-SUN communication device

Also, some words I didn't know came out.

Wi-SUN is a wireless standard used for communication with smart meters. (Not limited to smart meters, it is an international standard originating in Japan that aims for low power consumption, long distance and high reliability wireless standards that are impossible with Wi-Fi, but now it is a standard that is practically used only for Japanese smart meters I'm worried that Galapagos will be called in the future.)

Here, ROHM's BP35A1 is used as the Wi-SUN communication device. I will. It seems to be the cheapest Wi-SUN device that the average person can buy.

BP35A1withMathor.jpg

Looking at this picture, it seems that the motherboard BP359C is necessary, but it is unnecessary because it is full of unnecessary functions.

The adapter board BP35A7A is, in other words, a "breakout board". It is a guy who converts to a 2.54mm pitch that is easy to use for amateur electronic work.

Since the adapter board BP35A7A has a 3.3V level serial port, you can connect it to a microcomputer such as Raspberry Pi, or connect it to a PC through USB-serial conversion.

Easier

BP35A1, USB dongle with built-in USB-serial conversion, [WSR35A1-00](http://www.rohm.co.jp/web/japan/news-detail?news-title=2014-10-02_news&defaultGroupId = false) is also available. It is 15,000 yen (excluding tax).

WSR35A1-00.jpg

I think this is easy for beginners of electronic work. There is a drawback (?) That it is no longer an electronic work because it is just plugged into a USB port.

For those who can do electronic work, BP35A1, adapter board and USB-serial conversion (or directly connected to the serial pin of Raspberry Pi) will make it a little cheaper.

Hardware preparation summary

--Apply to use the B route service --Buy a Wi-SUN communication device (either of the following) --BP35A1 and adapter board (and connector screws (and USB serial conversion)) --WSR35A1-00 (recommended)

Also, I omitted it, but I also need a computer to be the mother ship. The sample below was written in python, so I think any computer running python will do.

software

There are about three materials needed to create software. You don't have to look at the sample in this article to move it, but if you want to make further improvements, please refer to it.

Documents for BP35A1

"BP35A1 Command Reference Manual" at "ROHM Sub-GHz Series" Support Page

Even if you use WSR35A1-00, the contents are BP35A1, so you will need to refer to the "BP35A1 Command Reference Manual".

ECHONET Lite standard

ECHONET Lite Standards Ver.1.12 (Japanese version)

Up to 5 copies are full of volume, but for the time being, I think it's enough to look at "Part 2 ECHONET Lite Communication Middleware Specifications".

ECHONET device object detailed regulations

APPENDIX ECHONET Device Object Detailed Regulations Release H

It's bigger than the main body for APPENDIX, and it's an appendix commercial law! It's in a state, but it's big because the story of various devices is written. Looking at the table of contents, you'll be thrilled to see all kinds of home appliances covered, right?

For the time being, it is OK to look only at the smart meter (low-voltage smart electric energy meter class regulation).

Sample program

As a program creation policy

--Priority is given to the ease of understanding of communication specifications --No error handling --Do not create utility functions so that they can be read straight

We attach great importance to such things. It's a badly behaved program.

The language used was Python 2.7. I am using pyserial for serial communication. In the case of Raspberry, Python and pyserial are included from the beginning (2014-06-02 or later)

It has been checked for operation on Windows and Mac. (Assuming that Python + pyserial is installed)

Edit the ID, password, and serial port device name according to your environment.

bp35a1.py


#!/usr/bin/env python
# -*- coding: utf-8 -*-

from __future__ import print_function

import sys
import serial
import time

#B route authentication ID (the one sent by mail from TEPCO Power Grid)
rbid  = "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
#B route authentication password (the one sent by e-mail from TEPCO Power Grid)
rbpwd = "XXXXXXXXXXXX"
#Serial port device name
serialPortDev = 'COM3'	#For Windows
serialPortDev = '/dev/ttyUSB0'	# Linux(In the case of Raspberry Pi etc.)
serialPortDev = '/dev/cu.usbserial-A103BTPR'	#For Mac

#Serial port initialization
ser = serial.Serial(serialPortDev, 115200)

#Try to get the version for the time being (you don't have to do it)
ser.write("SKVER\r\n")
print(ser.readline(), end="") #Echo back
print(ser.readline(), end="") #version

#B root authentication password setting
ser.write("SKSETPWD C " + rbpwd + "\r\n")
print(ser.readline(), end="") #Echo back
print(ser.readline(), end="") #OK should come (no check)

#B route authentication ID setting
ser.write("SKSETRBID " + rbid + "\r\n")
print(ser.readline(), end="") #Echo back
print(ser.readline(), end="") #OK should come (no check)

scanDuration = 4;   #Scan time. It's 6 in the sample, but you can go with 4. (If it doesn't work, increase it and try again)
scanRes = {} #Scan result container

#Scan retry loop (until something is found)
while not scanRes.has_key("Channel") :
	#Perform active scan (with IE)
	#It takes time. About 10 seconds?
	ser.write("SKSCAN 2 FFFFFFFF " + str(scanDuration) + "\r\n")

	#Loop to end scan for each scan
	scanEnd = False
	while not scanEnd :
		line = ser.readline()
		print(line, end="")

		if line.startswith("EVENT 22") :
			#I've finished scanning (whether found or not)
			scanEnd = True
		elif line.startswith("  ") :
			#If you scan and find it, open two spaces and the data will come
			#Example
			#  Channel:39
			#  Channel Page:09
			#  Pan ID:FFFF
			#  Addr:FFFFFFFFFFFFFFFF
			#  LQI:A7
			#  PairID:FFFFFFFF
			cols = line.strip().split(':')
			scanRes[cols[0]] = cols[1]
	scanDuration+=1

	if 7 < scanDuration and not scanRes.has_key("Channel"):
		#You can specify up to 14 as an argument, but if it fails with 7, it is useless any more
		print("Scan retry over")
		sys.exit()	####Thread winter end####

#Set Channel from the scan result.
ser.write("SKSREG S2 " + scanRes["Channel"] + "\r\n")
print(ser.readline(), end="") #Echo back
print(ser.readline(), end="") #OK should come (no check)

#Set Pan ID from scan results
ser.write("SKSREG S3 " + scanRes["Pan ID"] + "\r\n")
print(ser.readline(), end="") #Echo back
print(ser.readline(), end="") #OK should come (no check)

#MAC address(64bit)To an IPV6 link-local address.
# (I'm converting using the function of BP35A1, but is there a story that I should just convert the character string? ??)
ser.write("SKLL64 " + scanRes["Addr"] + "\r\n")
print(ser.readline(), end="") #Echo back
ipv6Addr = ser.readline().strip()
print(ipv6Addr)

#Starts the PANA connection sequence.
ser.write("SKJOIN " + ipv6Addr + "\r\n");
print(ser.readline(), end="") #Echo back
print(ser.readline(), end="") #OK should come (no check)

#Waiting for PANA connection completion (returns about 10 lines)
bConnected = False
while not bConnected :
	line = ser.readline()
	print(line, end="")
	if line.startswith("EVENT 24") :
		print("PANA connection failure")
		sys.exit()	####Thread winter end####
	elif line.startswith("EVENT 25") :
		#Connection completed!
		bConnected = True

#After that, set the serial communication timeout
ser.timeout = 2

#Smart meter throws instance list notification
# (ECHONET-Lite_Ver.1.12_02.pdf p.4-16)
print(ser.readline(), end="") #ignore

while True:
	#ECHONET Lite frame creation
	#Reference material
	#・ ECHONET-Lite_Ver.1.12_02.pdf (EL below)
	#・ Appendix_H.pdf (Below AppH)
	echonetLiteFrame = ""
	echonetLiteFrame += "\x10\x81"		# EHD (reference:EL p.3-2)
	echonetLiteFrame += "\x00\x01"		# TID (reference:EL p.3-3)
	#EDATA from here
	echonetLiteFrame += "\x05\xFF\x01"	# SEOJ (reference:EL p.3-3 AppH p.3-408～)
	echonetLiteFrame += "\x02\x88\x01"	# DEOJ (reference:EL p.3-3 AppH p.3-274～)
	echonetLiteFrame += "\x62"			# ESV(62:Property value read request) (reference:EL p.3-5)
	echonetLiteFrame += "\x01"			# OPC(1 piece)(reference:EL p.3-7)
	echonetLiteFrame += "\xE7"			# EPC(reference:EL p.3-7 AppH p.3-275)
	echonetLiteFrame += "\x00"			# PDC(reference:EL p.3-9)
	
	#Command transmission
	command = "SKSENDTO 1 {0} 0E1A 1 {1:04X} {2}".format(ipv6Addr, len(echonetLiteFrame), echonetLiteFrame)
	ser.write(command)

	print(ser.readline(), end="") #Echo back
	print(ser.readline(), end="") #EVENT 21 should come (unchecked)
	print(ser.readline(), end="") #OK should come (no check)
	line = ser.readline()         #ERX UDP should come
	print(line, end="")
	
	#The received data sometimes comes with different data
	#Because you may miss it and pick up strange data
	#The check is strict.
	if line.startswith("ERXUDP") :
		cols = line.strip().split(' ')
		res = cols[8]	#UDP received data part
		#tid = res[4:4+4];
		seoj = res[8:8+6]
		#deoj = res[14,14+6]
		ESV = res[20:20+2]
		#OPC = res[22,22+2]
		if seoj == "028801" and ESV == "72" :
			#Smart meter(028801)Response from(72)If
			EPC = res[24:24+2]
			if EPC == "E7" :
				#The content is an instantaneous power measurement value(E7)If so
				hexPower = line[-8:]	#The last 4 bytes (8 hexadecimal characters) are the instantaneous power measurements
				intPower = int(hexPower, 16)
				print(u"Instantaneous power measurement:{0}[W]".format(intPower))
				


#I won't come here because it's an infinite loop
ser.close()

See the comments for more information on the program.

When I do this, I get the following output: Since it is a sample for promoting understanding, the communication content is noisy.

$ ./bp35a1.py
SKVER
EVER 1.2.10
OK
SKSETPWD C XXXXXXXXXXXX
OK
SKSETRBID XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
OK
SKSCAN 2 FFFFFFFF 4
OK
EVENT 22 FE80:0000:0000:0000:XXXX:XXXX:XXXX:XXXX
SKSCAN 2 FFFFFFFF 5
OK
EVENT 20 FE80:0000:0000:0000:XXXX:XXXX:XXXX:XXXX
EPANDESC
  Channel:39
  Channel Page:09
  Pan ID:XXXX
  Addr:XXXXXXXXXXXXXXXX
  LQI:95
  PairID:XXXXXXXX
EVENT 22 FE80:0000:0000:0000:XXXX:XXXX:XXXX:XXXX
SKSREG S2 39
OK
SKSREG S3 XXXX
OK
SKLL64 XXXXXXXXXXXXXXXX
FE80:0000:0000:0000:XXXX:XXXX:XXXX:XXXX
SKJOIN FE80:0000:0000:0000:XXXX:XXXX:XXXX:XXXX
OK
EVENT 21 FE80:0000:0000:0000:XXXX:XXXX:XXXX:XXXX 00

~ Omitted ~

SKSENDTO 1 FE80:0000:0000:0000:XXXX:XXXX:XXXX:XXXX 0E1A 1 000E
EVENT 21 FE80:0000:0000:0000:XXXX:XXXX:XXXX:XXXX 00
OK
ERXUDP FE80:0000:0000:0000:XXXX:XXXX:XXXX:XXXX FE80:0000:0000:0000:XXXX:XXXX:XXXX:XXXX 0E1A 0E1A XXXXXXXXXXXXXXXX 1 0012 1081000102880105FF017201E704000002AE
Instantaneous power measurement:686[W]
SKSENDTO 1 FE80:0000:0000:0000:XXXX:XXXX:XXXX:XXXX 0E1A 1 000E
EVENT 21 FE80:0000:0000:0000:XXXX:XXXX:XXXX:XXXX 00
OK
ERXUDP FE80:0000:0000:0000:XXXX:XXXX:XXXX:XXXX FE80:0000:0000:0000:XXXX:XXXX:XXXX:XXXX 0E1A 0E1A XXXXXXXXXXXXXXXX 1 0012 1081000102880105FF017201E7040000029B
Instantaneous power measurement:667[W]

~ Repeat ~

The first negotiation takes about 10 seconds. Also, it seems that the response comes after about 1 second after throwing the request. I have the impression that it is a little slow for "real time" and "instantaneous value".

in conclusion

We introduced the hardware BP35A1 for communicating with smart meters and provided a minimal sample of software that uses it. It looks like a program that only outputs to standard output, but I think it was useful for understanding the communication specifications because there was no extra code such as for GUI.

Smart meters can only be used by connecting them to various devices, so I think it is very valuable for general users to be able to access raw information. We hope that this article will help you apply it to your home hack.

Addendum 1

I have omitted the referenced site. Excuse me.

View the instantaneous power and history of smart meters with a web browser

[I made a power meter with a smart meter and a Raspberry Pi] (http://chappnet.hateblo.jp/entry/2015/08/23/112738)

Addendum 2

The response format of ERXUDP (and ERXTCP) is binary by default, but it is awkward and needs to be changed to hexadecimal ASCII.

Before running the sample Python code, let's send the WOPT 01 command with ** TeraTerm etc. ** **

It is not recommended to send it programmatically at startup as it will write to the built-in flash the moment you type this command. When embedding it in a program, use the ROPT command to read the current settings and send the WOPT 01 command only when changes are needed.

Addendum 3

It is controversial whether the price of the USB dongle WSR35A1-00 is 15,000 yen, or the price of BP35A1 and the surroundings is less than 10,000 yen, which is cheap or expensive. It's a catchy title, but I personally think it's expensive. I think that if we can get more devices, we can get closer to the price of a WiFi dongle (about 1000 yen). If you know of a cheaper device, please let us know in the comments. (It seems that there is a FLET'S Mill Energy 5000 yen USB dongle, but I don't know the specifications, so I can't use it unless I sign up for FLET'S Mierne.)

Addendum 4

A product called RL7023 Stick-D / IPS has been released by Tessera Technology Co., Ltd. You can purchase it from chip1stop for 8,060 yen (excluding tax). The communication specifications seem to be the same as WSR35A1-00 and BP35A1. I confirmed that the above program works as it is. It's small and cheap, so the current recommendation is here.

DSC_0447.JPG

Addendum 5

A Wi-SUN HAT kit for easily connecting the M5StickC and BP35A1 is available at BOOTH. "Wi-SUN HAT" kit for M5 Stick C If you want to use the ESP32 series, I think this combination is the best buy. (If you want to use it with a USB-equipped device such as Raspberry Pi, I think the RL7023 Stick-D / IPS in the previous section is good.)

Recommended Posts

Pull out smart meter information with the cheapest hardware
Read the power of the smart meter with M5StickC (BP35C0-J11-T01)
Get the latest AMI information with the AWS CLI
Find out the day of the week with datetime