The guidance of the head family is here. I understand that it is easy to implement distributed processing of batches, which is difficult when assembled properly.
$ brew install zookeeper
$ ls /usr/local/etc/zookeeper
defaults log4j.properties zoo.cfg zoo_sample.cfg
Even if you compare the setting file that is placed by default with Tutorial, it is almost the same, so start it as it is Try.
$ cat /usr/local/etc/zookeeper/zoo.cfg | egrep -v "#"
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/usr/local/var/run/zookeeper/data
clientPort=2181
$ which zkServer
/usr/local/bin/zkServer
$ sudo zkServer start
ZooKeeper JMX enabled by default
Using config: /usr/local/etc/zookeeper/zoo.cfg
Starting zookeeper ... STARTED
The log is output below
$ tail -f /usr/local/var/log/zookeeper/zookeeper.log
2017-01-09 06:40:05 QuorumPeerMain [WARN] Either no config or no quorum defined in config, running in standalone mode
Something is out. Quorum refers to the destination server, a warning that it does not exist. It's a trial stand-alone here, so it doesn't matter if it's out.
You can connect to the port specified as clientPort in the configuration file.
$ cat /usr/local/etc/zookeeper/zoo.cfg | grep clientPort
clientPort=2181
Since port 2181 is specified, connect to it.
$ which zkCli
/usr/local/bin/zkCli
$ sudo zkCli -server 127.0.0.1:2181
Connecting to 127.0.0.1:2181
Welcome to ZooKeeper!
JLine support is enabled
WATCHER::
WatchedEvent state:SyncConnected type:None path:null
[zk: 127.0.0.1:2181(CONNECTED) 0]
[zk: 127.0.0.1:2181(CONNECTED) 0] help
ZooKeeper -server host:port cmd args
stat path [watch]
set path data [version]
ls path [watch]
delquota [-n|-b] path
ls2 path [watch]
setAcl path acl
setquota -n|-b val path
history
redo cmdno
printwatches on|off
delete path [version]
sync path
listquota path
rmr path
get path [watch]
create [-s] [-e] path data acl
addauth scheme auth
quit
getAcl path
close
connect host:port
Follow the tutorial (http://oss.infoscience.co.jp/hadoop/zookeeper/docs/r3.3.1/zookeeperStarted.html). znode refers to each node on the ZooKeeper tree.
#Directory check
[zk: 127.0.0.1:2181(CONNECTED) 0] ls /
[zookeeper]
#node(znode)Create
[zk: 127.0.0.1:2181(CONNECTED) 1] create /test_node my_test_data
Created /test_node
#Creation confirmation
[zk: 127.0.0.1:2181(CONNECTED) 2] ls /
[test_node, zookeeper]
#Make sure the data you created is associated with the node
[zk: 127.0.0.1:2181(CONNECTED) 3] get /test_node
my_test_data
cZxid = 0x8
ctime = Mon Jan 09 07:00:17 JST 2017
mZxid = 0x8
mtime = Mon Jan 09 07:00:17 JST 2017
pZxid = 0x8
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 12
numChildren = 0
#Try changing the associated data
[zk: 127.0.0.1:2181(CONNECTED) 4] set /test_node replaced_my_test_data
cZxid = 0x8
ctime = Mon Jan 09 07:00:17 JST 2017
mZxid = 0x9
mtime = Mon Jan 09 07:03:21 JST 2017
pZxid = 0x8
cversion = 0
dataVersion = 1
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 21
numChildren = 0
#Verification
[zk: 127.0.0.1:2181(CONNECTED) 5] get /test_node
replaced_my_test_data # <-changed
cZxid = 0x8
ctime = Mon Jan 09 07:00:17 JST 2017
mZxid = 0x9
mtime = Mon Jan 09 07:03:21 JST 2017
pZxid = 0x8
cversion = 0
dataVersion = 1
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 21
numChildren = 0
#Delete
[zk: 127.0.0.1:2181(CONNECTED) 6] delete /test_node
#Verification
[zk: 127.0.0.1:2181(CONNECTED) 7] ls /
[zookeeper]
python x zookeeper uses a library called kazoo. I will write various things according to the following documents. kazoo
installation of kazoo
$ pip install kazoo
Roughly try
hello_kazoo.py
# coding:utf-8
from kazoo.client import KazooClient
from kazoo.client import KazooState
from kazoo.client import KeeperState
from kazoo.handlers.gevent import SequentialGeventHandler
import logging
logging.basicConfig()
#znode root
root = '/root'
#zookeeper client
zk = KazooClient(hosts='127.0.0.1:2181', read_only=True, handler=SequentialGeventHandler())
#Start in asynchronous mode
event = zk.start_async()
event.wait(timeout=3)
#Suspend processing if unable to connect to zookeeper server
if not zk.connected:
zk.stop()
raise Exception("Unable to connect.")
def listener(state):
'''
Listener when changing State
'''
print('current state is ' + state)
zk.add_listener(listener)
@zk.add_listener
def watch_for_ro(state):
if state == KazooState.CONNECTED:
if zk.client_state == KeeperState.CONNECTED_RO:
print('state is read_only')
else:
print('state is writable')
def print_status(znode):
'''
Get node status
'''
print('#####[DEBUG]#####')
#Check version and registration data
data, stat = zk.get(znode)
print('Version: %s, data: %s' % (stat.version, data.decode('utf-8')))
#Get a list of root child nodes
children = zk.get_children(root)
print("There are %s children with names %s" % (len(children), children))
def doAsync(async_obj):
'''
Asynchronous callback function(There is no particular meaning in the processing content)
'''
znodes = async_obj.get()
try:
children = async_obj.get()
#Output the names of all child nodes
print('#####[print child znodes]#####')
for child in children:
print(child)
except (ConnectionLossException, NoAuthException):
print("ERROR!!!")
sys.exit(1)
if __name__ == '__main__':
#Transaction start
tx = zk.transaction()
##Check the basic usage
#Path generation
zk.ensure_path(root)
#Create if znode has not been created
znode = root + '/sample_znode'
if zk.exists(znode) is None:
zk.create(znode, b'sample_data')
print_status(znode)
#Data update
zk.set(znode, b'updated_data')
print_status(znode)
#Add child node
znode2 = root + '/sample_znode2'
if zk.exists(znode2) is None:
zk.create(znode2, b'sample_data2')
print_status(znode2)
##Asynchronous processing is used like this
async_obj = zk.get_children_async(root)
async_obj.rawlink(doAsync)
#Delete node
zk.delete(root, recursive=True)
#commit
results = tx.commit()
print('#####[Result]#####')
print(results)
Execution result
$ python hello_kazoo.py
#####[DEBUG]#####
Version: 0, data: sample_data
There are 1 children with names ['sample_znode']
#####[DEBUG]#####
Version: 1, data: updated_data
There are 1 children with names ['sample_znode']
#####[DEBUG]#####
Version: 0, data: sample_data2
There are 2 children with names ['sample_znode', 'sample_znode2']
#####[print child znodes]#####
sample_znode
sample_znode2
#####[Result]#####
[]
$ sudo zkCli -server 127.0.0.1:2181
You can check various movements by connecting to the ZooKeeper server with and writing the code little by little while checking.
Recommended Posts