[LINUX] Send log data from the server to Splunk Cloud


I had the opportunity to use Splunk Cloud. Therefore, I will leave a memorandum on how to transfer various logs from the Linux server.

This article is based on Get * nix data into Splunk Cloud

Advance preparation

-Register in Splunk Cloud --Free trial is okay --Prepare an instance of Linux --We have confirmed the operation with Wordpress in the GCP Marketplace.


  1. Install Universal Forwarder on the server
  2. Download and install Universal Forwarder credentials
  3. Install Add-on for Linux on Universal Forwarder
  4. Install the App to parse the Add-on installed in 3.
  5. Confirm that the data can be received

1. Install Universal Forwarder on the server

After logging in to splunk.com, download the Universal Forwarder from here (https://www.splunk.com/en_us/download/universal-forwarder.html). image.png Click Download Now for your favorite package. After accepting the terms, you can get commands that can be executed from within the instance by pressing Download via Command Line (wget).

$ cd /tmp
$ wget ... #The command you got earlier
#Get the package that suits your environment
$ sudo rpm -i splunkforwarder-<…>-linux-2.6-x86_64.rpm 
$ sudo dpkg -i splunkforwarder-<…>-linux-2.6-x86_64.deb
#After installation, the splunk user will be added
$ sudo su - spulnk
$ cd /opt/splunkforwarder/bin
$ ./splunk start
#You will be prompted for your username and password.
#splunk because it is used by splunk forwarder.It's okay with something different from com
#Exit when the following is displayed
All installed files intact.
All preliminary checks passed.
Starting splunk server daemon (splunkd)...

2. Download and install Universal Forwarder credentials

  1. Go to https: // <your instance> .splunkcloud.com / ja-JP / app / splunkclouduf / setupuf image.png
  2. Download the credentials from 3. Download your customized universal forwarder credentials package.
  3. Transfer the downloaded credentials to your instance via sftp etc.
$ pwd
$ ls
$ sudo su - splunk
$ /opt/splunkforwarder/bin/splunk install app <PATH-TO-FILE>/splunkclouduf.spl
#If you are prompted for a username and password, enter the one you just mentioned

#Installation is complete when the following message is displayed.
App '<PATH-TO-FILE>/splunkclouduf.spl' installed

#Restart the universal forwarder
$ ./splunk restart

3. Install Add-on for Linux on Universal Forwarder

You can extend the functionality by downloading the Add-on from Splunkbase and installing it in the forwarder. This time, we will introduce Splunk Add-on for Unix and Linux. By introducing this Add-on, you can set the transfer of various logs in the Linux environment.

  1. Download the Splunk Add-on for Unix and Linux (https://splunkbase.splunk.com/app/833/).

Work in the terminal below

# /Downloads/splunk-add-on-for-unix-and-linux_602.Add to tgz-Suppose on is downloaded

#Copy the tar file downloaded to the instance using scp etc.
$ scp splunk@<LINUX_INSTANCE_IP_ADDR>:/tmp/ /Downloads/splunk-add-on-for-unix-and-linux_602.tgz

#Log in to your Linux instance. Below this, work with your Linux instance.
$ cd /tmp
$ tar xvfz splunk-add-on-for-unix-and-linux_602.tgz

# Splunk_TA_Make sure you have a directory called nix
$ ls

# splunkforwarder/etc/Move to apps
$ mv Splunk_TA_nix/ /opt/splunkforwarder/etc/apps
$ cd /opt/splunkforwarder/etc/apps/Splunk_TA_nix

#Copy the config file
$ mkdir local
$ cp default/inputs.conf local

#setting file
$ cat local/inputs.conf | head -n 6
# Copyright (C) 2020 Splunk Inc. All Rights Reserved.
interval = 60
sourcetype = vmstat
source = vmstat
disabled = 1

# disabled =Data can be transferred by setting it to 0.
$ sed -e 's/disabled = 0/disabled = 1/g' local/inputs.conf > local/inputs.conf
$ sed -e 's/disabled = true/disabled = false/g' local/inputs.conf > local/inputs.conf

#After making changes, reboot
$ cd /opt/splunkforwarder/bin
$ ./splunk restart

4. Install the App to parse the Add-on installed in 3.

After completing the above settings, the data from Linux has already been sent. However, some of the data has not been parsed, so it is difficult to use as it is. That's why we're introducing the Splunk Add-on for Unix and Linux to Splunk Cloud. Data can be parsed by introducing this Add-on.

  1. From the top left of Splunk Cloud, select `Search for other apps`` image.png
  2. Search for Splunk Add-on for Unix and Linux and install it.

5. Confirm that the data can be received

image.png If the settings are correct, the top result will be parsed and displayed like this.

Reference link

Get *nix data into Splunk Cloud

Recommended Posts

Send log data from the server to Splunk Cloud
Send a message from the server to your Chrome extension using Google Cloud Messaging for Chrome
[python] Send the image captured from the webcam to the server and save it
Send a message from Slack to a Python server
Log in to the remote server with SSH
POST images from ESP32-CAM (MicroPython) to the server
The guy who gets fitbit data from the server
The one that graphs the one that got the fitbit data from the server
Send data from Python to Processing via socket communication
Log in to the fortigate (6.0) management screen from selenium-try to log out
[Postgresql] SSH connection to the external DB server from the client
How to log in automatically like 1Password from the CLI
[Introduction to matplotlib] Read the end time from COVID-19 data ♬
Build a Python environment and transfer data to the server
The story of copying data from S3 to Google's TeamDrive
Pass OpenCV data from the original C ++ library to Python
I tried to output the access log to the server using Node.js
Start data science on the cloud
Send commands from Atom to Maya
Terminal association from the server side to Amazon SNS (python + boto3)
I want to send a signal only from the sub thread to the main thread
Try using the Python web framework Django (1)-From installation to server startup
Implemented in Dataflow to copy the hierarchy from Google Drive to Google Cloud Storage
Copy data from Amazon S3 to Google Cloud Storage with Python (boto)
How to build an application from the cloud using the Django web framework
Is it possible to extract the person's profile information from the chat log?
SIGNATE Quest ① From data reading to preprocessing
How to set the server time to Japanese time
Send a message from Python to Slack
Push notification from Python server to Android
Send data to DRF API with Vue.js
How to operate Linux from the console
How to access the Datastore from the outside
Visualize the export data of Piyo log
[LINE Messaging API] I want to send a message from the program to everyone's LINE
Use PIL in Python to extract only the data you want from Exif
[IBM Cloud] I tried to access the Db2 on Cloud table from Cloud Funtions (python)
SSH login to the target server from Windows with a click of a shortcut
[Free] Hit the Clash Royale API from lambda and send it to LINE
The first step to log analysis (how to format and put log data in Pandas)
[Kaggle] From data reading to preprocessing and encoding
Setting to output the log of cron execution
Just add the python array to the json data
The inaccuracy of Tensorflow was due to log (0)
I searched for railway senryu from the data
I tried to save the data with discord
[Python] How to read data from CIFAR-10 and CIFAR-100
GCP: Repeat from Pub / Sub to Cloud Functions, Cloud Functions to Pub / Sub
Send data from Raspberry Pi using AWS IOT
How to call Cloud API from GCP Cloud Functions
Data preprocessing (2) Data is changed from Categorical to Numerical.
Change the decimal point of logging from, to.
How to use the Google Cloud Translation API
Data retrieval from MacNote3 and migration to Write
How to operate Linux from the outside Procedure
[Python] Flow from web scraping to data analysis
How to measure line speed from the terminal
Download the VGG Face2 dataset directly to the server
[Python] I will upload the FTP to the FTP server.
Use pygogo to get the log in json.
From the introduction of pyethapp to the execution of contract