Let's write the blackbird plugin The third bullet, where I could not write in detail in the second bullet, mainly
Let's die.
JobBase classself.options
I explained this in the second bullet, but interpret the contents described in the config file as key and value, It is in dictionary format. In particular
[SECTION_NAME_IS_ANYTHING_OK]
hostname = hogehoge.com
hogehoge = fugafuga
If there is a config file like
self.options = {
'hostname': 'hogehoge.com',
'hogehoge': 'fugafuga'
}
You will have a dictionary that looks like this. I intend to write about this in quite detail in the second installment, so It is like this. More kwsk! If there is something like that, I think I'll write it in the comments or if you contact me.
self.queue
The Main process is a Queue for exchanging messages between Threads. I didn't do anything particularly difficult
I'm using the Python-made Queue.Queue as it is.
So think of the self.queue.put method as being the same as reading Queue.Queue.put.
notes:
Regarding the put method (I think that plugin writers will mainly use the put method)
When there is an argument called block and you are doing a put operation, you can perform operations from other Consumers and Producers.
It's about blocking.
However, this is under the condition that key, clock, host do not cover one item.
** block = False is highly recommended **.
self.logger
An instance for logging. Since this instance is just a Python-made logging module with only the output format changed, The methods of the logging module can be used as they are.
self.logger.debut('This is Debug log!! HOGEHOGE!!!!!!!!!!')
Therefore, it is also possible to write as above. The log output level is WARN by default (this can be changed in the global section).
self.invalid_key_list
//It's new, but I feel like I made a mistake in the property name.
//I feel like it's not invalid w ignore??
This is a blacklist that if the Item key exists in this list, it cannot be put in the Queue. For example
class ConcreteJob(blackbird.plugins.base.JobBase):
def __init__(self, options, queue=None, logger=None):
super(ConcreteJob, self).__init__(options, queue, logger)
self.invalid_key_list = [
'host.IGNORE_KEY_NAME001',
'host.IGNORE_KEY_NAME002'
]
By doing so, it is possible to cut out the logic to another method without writing the distribution process in the place where it is fetched completely like the stats command of a certain memcached.
In this case, of course, you can't just add it to the list.
#..snip..
def filter_item(item):
if not item.data['key'] in self.invalid_key_list:
return item
def filter_items(items):
filtered_items = list()
for entry in items:
if not items.data.['key'] in self.invalid_key_list:
filtered_items.append(entry)
return filtered_items
I hope you can implement it with a good feeling.
Next, I would like to talk about the convenience functions implemented on each base class side. Well, to be honest, there aren't many, but I think they will help you create plugins.
blackbird.plugins.base.ValidatorBase.detect_hostname
As the name suggests, it is a method that fetches the hostname. I'm using low-layer functions (I hope Python can use this kind of thing lightly), but it's really simple,
return socket.getfqdn() or socket.gethostname() or 'localhost'
I'm just doing it.
hostname -s)localhost as a string.As a messenger
class Validator(base.ValidatorBase):
def __init__(self):
self.__spec = None
@property
def spec(self):
self.__spec = (
"[{0}]".format(__name__),
"hostname = string(default={0})".format(self.detect_hostname()),
)
return self.__spec
Is it such a place? By the way, this was implemented at the suggestion of a big senior, and thank you for taking this opportunity. M (\ _ \ _) m
Timer context
Timer context is a context that measures the processing time in the context enclosed in with. We use it to write and read keys to redis and memcached to measure ratency and make it Metric. The usage is no different from ordinary Context,
import blackbird.plugins.base.Timer
with Timer() as timer:
for entry in xrange(0, 9):
print entry
return [timer.sec, timer.msec]
In other words, the sec property of the timer class returns the processing time in seconds. The msec property returns the processing time in milliseconds.
Exception (the real meaning here is Exception when an unexpected error occurs in your plugin)
To distinguish whether it was unexpected when it occurred or was intentionally generated like a raise Value Error.
It implements its own Exception.
However, since you only need to specify the error string and raise it, raise it in exactly the same way as ValueError. For example
import blackbird.plugins.base.BlackbirdPluginError
try:
value = self.options['key']
except KeyError as exception:
raise blackbird.plugins.base.BlackbirdPluginError(
exception.__str__
)
This makes it easier to debug the behavior when an Exception is thrown.
blackbird.plugins.base This is what the module does, but
I don't know what this is, or it's different here, and I think I want a method like this on the base side as well.
If there is, I would like to know more and more m (\ _ \ _) m