[PYTHON] Create a command to encode / decode Splunk base64

I think that the template was created last time, so let's create a command that handles Base64 that seems to be used.

There is talk that nkf is enough. : sweat:

@uneyamauneko-san, could you use python3?

Code validation with Python

base64encode.py

import base64 as b64
message='test message is utf-8 text'
message.encode()
enc_text=b64.b64encode(message.encode())
enc_text.decode()

Output 'dGVzdCBtZXNzYWdlIGlzIHV0Zi04IHRleHQ ='

base64decode.py

enc_message='dGVzdCBtZXNzYWdlIGlzIHV0Zi04IHRleHQ='
dec_text=b64.b64decode(enc_message.encode())
dec_text.decode()

Output 'test message is utf-8 text'

As you can see, Python's base64 handles bytes, so you need to doencode () `` decode ()as appropriate.

Commands.conf

commands.conf

[base64]
chunked = true
filename = base64encdec.py

Place it in $ SPLUNK_HOME/etc/<< APPS >>/default /. ** Filename should not be with the module to import ** I was pretty addicted to base64.py ...: cry:

Code

base64encdec.py

#!/usr/bin/env python

import sys, base64
from splunklib.searchcommands import dispatch, StreamingCommand, Configuration, Option, validators

@Configuration()
class base64Command(StreamingCommand):
    """ Base64 encode and decode text

    ##Syntax

    .. code-block::
        base64 output=<field> action=<enc|dec> <field>

    ##Description
    Outputs the string of the input field with base64 encoding/decoding.

    ##Example
    Encode

    .. code-block::
        | makeresults | base64 output=enc_time action=enc _time

    Decode

    .. code-block::
        | makeresults | eval enc_text = "cHl0aG9uIGlzIGRpZmZpY3VsdCBmb3IgbWU=" | base64 output=plain_text action=dec enc_text

    """

    output = Option(
        doc='''
        **Syntax:** **output=***<fieldname>*
        **Description:** Name of the field that will hold the output text''',
        require=True, validate=validators.Fieldname())

    action = Option(
        doc='''
        **Syntax:** **action=***<enc|dec>*
        **Description:** Name of the action in encoding/decoding''',
        require=True, validate=validators.Set('enc','dec'))


    def stream(self, records):
        self.logger.debug('base64Command: %s', self)  # logs command line
        for record in records:
            for fieldname in self.fieldnames:
                pass
            if self.action == "enc":
                record[self.output]=base64.b64encode(record[fieldname].encode()).decode()
            else:
                record[self.output]=base64.b64decode(record[fieldname].encode()).decode()
            yield record

dispatch(base64Command, sys.argv, sys.stdin, sys.stdout, __name__)

Place it in $ SPLUNK_HOME/etc/<< APPS >>/bin /.

#!/usr/bin/env python

import sys, base64
from splunklib.searchcommands import dispatch, StreamingCommand, Configuration, Option, validators

@Configuration()
class base64Command(StreamingCommand):
    """ Base64 encode and decode text

    ##Syntax

    .. code-block::
        base64 output=<field> action=<enc|dec> <field>

    ##Description
    Outputs the string of the input field with base64 encoding/decoding.

    ##Example
    Encode

    .. code-block::
        | makeresults | base64 output=enc_time action=enc _time

    Decode

    .. code-block::
        | makeresults | eval enc_text = "cHl0aG9uIGlzIGRpZmZpY3VsdCBmb3IgbWU=" | base64 output=plain_text action=dec enc_text

    """

    output = Option(
        doc='''
        **Syntax:** **output=***<fieldname>*
        **Description:** Name of the field that will hold the output text''',
        require=True, validate=validators.Fieldname())

    action = Option(
        doc='''
        **Syntax:** **action=***<enc|dec>*
        **Description:** Name of the action in encoding/decoding''',
        require=True, validate=validators.Set('enc','dec'))


    def stream(self, records):
        self.logger.debug('base64Command: %s', self)  # logs command line
        enc = lambda x: base64.b64encode(x.encode()).decode()
        dec = lambda x: base64.b64decode(x.encode()).decode()
        for record in records:
            record[self.output]=enc(record[self.fieldnames[0]]) if self.action == "enc" else dec(record[self.fieldnames[0]])
            yield record

dispatch(base64Command, sys.argv, sys.stdin, sys.stdout, __name__)

I tried various things to see if it could be included, but in the end I gave up.

for loop

  for record in records:
      
       yield record

In this process, each event is processed and output sequentially.

I could write (x in record for records), but I couldn't because I had to output it in a loop after all. I tried using the * lambda function *, so I wrote it as neatly as possible.

Trial

base64enc_dec.spl

| makeresults  
| base64 output=e_time action=enc _time 
| base64 output=d_time action=dec e_time

Commentary

Option This time I tried using validators.Set (). No information at all. : cry: I looked at Github code, validators.py and tried to move it, and it worked.

However, except for enc and dec, 2 errors occurred while the search was executing. Therefore, search results might be incomplete.

• A value for "action" is required
• Illegal value: action=de

It's only subtle, so it's subtle. I have to do searchbnf.conf properly.

if Since the evaluation of options is solid, the judgment of action is two choices Once you make a one-liner, you can make it the same shape as last time.

Maybe better

Command by George Starcher found in Slack

base64decode.py

import csv
import sys, os
if sys.argv[0] == '':
    mypath='.'
else:
    mypath=os.path.dirname(sys.argv[0])
sys.path.append(os.path.join(mypath,'lib'))
try:
    import base64
except ImportError as e:
    raise(e)
def decode_value(value):
    value_acsii = ""
    try:
        base64_bytes = value.encode('ascii')
        message_bytes = base64.b64decode(base64_bytes)
        value_ascii = message_bytes.decode('ascii')
    except Exception as e:
        pass
    return(value_ascii)
def main():
    if len(sys.argv) != 2:
        print("Usage: python external_b64decode.py [b64 field]")
        sys.exit(1)
    valuefield = sys.argv[1]
    infile = sys.stdin
    outfile = sys.stdout
    r = csv.DictReader(infile)
    header = r.fieldnames
    w = csv.DictWriter(outfile, fieldnames=r.fieldnames)
    w.writeheader()
    for result in r:
        value = result.get(valuefield)
        value_ascii = decode_value(value)
        result['value_ascii'] = value_ascii
        w.writerow(result)
main()

This makes it clear that you can create commands without the Splunk python SDK.

Summary

Regarding base64, there are apps such as DECRYPT, so I'm feeling better now. However, there are many things that can be understood by writing the code, so I will do it as a study.

https://github.com/bentleymi/ta-webtools It seems to be quite new

.conf17 slide introduced how to create a custom search command.