[PYTHON] Create a custom search command in Splunk (Streaming Command)

Reverse strings in Splunk. near the end

If you use it often, you may want to have an external python script.

I wrote that, so I made it. It was really hard: sweat_smile:

@uneyamauneko @ msi's article really helped me.

The copy source is countmatches.py@splunk-sdk-python

Preparation

You can do pip install splunk-sdk in Install the Splunk Enterprise SDK for Python.

But my environment

Environment variables, etc.

% uname -v 
Darwin Kernel Version 19.6.0: Tue Nov 10 00:10:30 PST 2020
% which python
/opt/anaconda3/bin/python
% python --version
Python 3.7.6
% $SPLUNK_HOME/bin/splunk --version
Splunk 8.1.1 (build 08187535c166)

With pip install splunk-sdk

pip result

 % pip install splunk-sdk
Requirement already satisfied: splunk-sdk in /opt/anaconda3/lib/python3.7/site-packages (1.6.14)

It is included. Certainly, I remember brew install splunk-sdk.

Here, set commands.conf in $ SPLUNK_HOME/etc/<< MYAPPS >>/default/as shown in Example. Https://docs.splunk.com/Documentation/Splunk/8.1.1/Admin/Commandsconf Is incorrect. Reported.

commands.conf

[rev]
chunked = true
filename = rev.py

important subjects

Create a link in / Applications/Splunk/lib/python3.7/site-packages.

ln -s /Applications/Splunk/lib/python3.7/site-packages $SPLUNK_HOME/lib/python3.7/site-packages/splunklib 

result

% pwd
/Applications/Splunk/lib/python3.7/site-packages
% ls -la
lrwxr-xr-x    1 XXXXX  wheel     52 12 31 10:53 splunklib -> /opt/anaconda3/lib/python3.7/site-packages/splunklib

Reason

--No matter how many times I try, import splunk lib does not work -- import splunklib works fine with python started with / usr/bin/env splunk. --import splunklib fails with python in $ SPLUNK_HOME/bin/splunk cmd python. ――It can't be helped, so I created a symbolic link and it worked.

Code

rev.py

#!/usr/bin/env python

import sys
from splunklib.searchcommands import dispatch, StreamingCommand, Configuration, Option, validators

@Configuration()
class reverseCommand(StreamingCommand):
    """ Reverse the string.

    ##Syntax

    .. code-block::
        rev output=<field> <field>

    ##Description
    Outputs the string of the input field in reverse order.

    ##Example

    .. code-block::
        index=_internal clientip=* | head 1 | rev output=r_clientip clientip

    """

    output = Option(
        doc='''
        **Syntax:** **output=***<fieldname>*
        **Description:** Name of the field that will hold the revesed text''',
        require=True, validate=validators.Fieldname())

    def stream(self, records):
        self.logger.debug('revCommand: %s', self)  # logs command line
        for record in records:
            for fieldname in self.fieldnames:
                pass
            record[self.output]=record[fieldname][-1::-1]
            yield record

dispatch(reverseCommand, sys.argv, sys.stdin, sys.stdout, __name__)

How to use

| rev output=<output field> <field>

result

rev.spl

index=_internal clientip=* | head 1
| rev clientip output=clientip_s

A difficult story named commentary

--Almost the original copy and paste -Shebang has no effect (likely).

First I put commands.conf and rev.py in place (default and bin) and restarted Splunk. At that time, such a file.

rev.py

#!/usr/bin/env python

import sys
from splunklib.searchcommands import dispatch, StreamingCommand, Configuration, Option, validators

@Configuration()
class reverseCommand(StreamingCommand):
    """ Reverse the string.
    """
    def stream(self, records):
        self.logger.debug('revCommand: %s', self)  # logs command line
        for record in records:
            yield record

dispatch(reverseCommand, sys.argv, sys.stdin, sys.stdout, __name__)

A command that does nothing. For the time being, I confirmed the operation with this. Once I've confirmed that | rev client ip works without error.

After that, I made various changes, executed it, saw the result, corrected it, and repeated the execution.

The main processing of this command is as follows.

main

    def stream(self, records):
        self.logger.debug('revCommand: %s', self)  # logs command line
        for record in records:
            for fieldname in self.fieldnames:
                pass
            record[self.output]=record[fieldname][-1::-1]
            yield record

Is it the Streaming Command that returns with yield without returning the return value with return?

After trying various things, it seems that the field name that is the argument of the command can only be taken with the fieldname created byfor fieldname in self.fieldnames:. I get the error unhashable type:'list'.

I tried merging the

    def stream(self, records):
        self.logger.debug('revCommand: %s', self)  # logs command line
        for record in records:
            record[self.output]=record[''.join(self.fieldnames)][-1::-1]
            yield record

Click here for reference StreamingCommand.fieldnames

If you simply overwrite record [fieldname] = record [fieldname] [-1 :: -1] is fine, but I think that the usability is subtle, so I set the option and changed to the above

Summary

It was very difficult because I made it for the first time. It's been a while since I couldn't import ...

Looking at the * Job Survey *, I feel that it is taking some time. For the time being, I think that you can make this as a template with python that simply returns.