[PYTHON] A script that transfers tweets containing specific Twitter keywords to Slack in real time

Learn how to monitor Twitter and forward tweets containing specific keywords to Slack in real time. The script is python3, and the Twitter API statuses / filter.json is used to get tweets in real time.

System schematic

Imagine this system configuration. This article provides an example script for twitter_monitoring.py at the bottom left of this figure.

The behavior of the script is to get tweets containing specific keywords with the Twitter API statuses / filter.json and have Slackbot post the tweets to any channel via the Slack API.

Advance preparation

Get ready to use the Twitter API and Slack API.

Create a Twitter API account

Token to be acquired

It's OK if you create a Twitter API account and get the following 4 tokens.

• API key
• API secret key
• Access token
• Access token secret

How to make

For how to create an account, the following site was helpful.

• https://www.itti.jp/web-direction/how-to-apply-for-twitter-api/
• https://dev.classmethod.jp/articles/twitter-api-approved-way/
• https://cre8cre8.com/python/twitter-api.htm

Creating a Slackbot

From the Slack API page, create a bot that will be the user for posting tweets.

Token to be acquired

After creating Slackbot, it is OK if you can get the following tokens from the ʻOAuth & Permissions` page of the Slack API management site.

• Bot User OAuth Access Token

How to make

For how to create Slackbot, the following site was helpful.

• https://techacademy.jp/magazine/27979
• https://belltree.life/slack-bot-hands-on/

Scope (authority)

For the scope (permission) setting of Slackbot, it is enough to allow only chat: write for this purpose.

Python script

I will explain about the twitter_monitoring.py created this time.

Whole code

Below is a coding example of a Python script (45 lines in total).

twitter_monitoring.py

# coding: utf-8
import json
import logging
from time import sleep

import requests_oauthlib
import slack

# Parameters
keyword = 'keyword' #Specify any search keyword

# Twitter parameters
consumer_key = 'xxxxxxxxxxxxxxxxxxxxxxxxx' # Twitter API key
consumer_secret = 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx' # Twitter API secret key
access_token = '9999999999999999999-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx' # Twitter Access token
access_token_secret = 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx' # Twitter Access token secret

# Slack parameters
bot_token = "xoxb-999999999999-9999999999999-xxxxxxxxxxxxxxxxxxxxxxxx" # Slack Bot User OAuth Access Token
channel_id = 'xxxxxxxxxxx' # Slack channel ID

# Logging
logging.basicConfig(filename='twitter_monitoring.log', format='%(asctime)s %(levelname)-8s %(message)s', datefmt='%Y-%m-%d %H:%M:%S', level=logging.WARNING)

#processing
uri = 'https://stream.twitter.com/1.1/statuses/filter.json'
twitter_session = requests_oauthlib.OAuth1Session(consumer_key, consumer_secret, access_token, access_token_secret)
slack_client = slack.WebClient(token = bot_token)

while True:
    try:
        logging.info("Request to Twitter API.")
        twitter_response = twitter_session.post(uri, data=dict(track=keyword), stream=True)
        if twitter_response.status_code != 200: #If you get caught in Twitter rate limit, etc.
            logging.warning("status_code:%s, reason:%s", twitter_response.status_code, twitter_response.reason)
            sleep(900) #Wait 15 minutes
            continue
        for line in twitter_response.iter_lines():
            if not line: #No corresponding tweet
                logging.info("No any tweets.")
                continue
            tweet_dict = json.loads(line.decode("utf-8"))
            tweet_link = 'https://twitter.com/i/web/status/' + tweet_dict['id_str']
            slack_response = slack_client.chat_postMessage(channel = channel_id , text = tweet_link) #Forward tweet links to Slack
    except Exception as e:
         logging.error(f'{e}')

Setting parameters

Since the code itself is short, this example embeds the configuration parameters in the script. Specify the values of various tokens and parameters in the variables below.

[^ 1]: The end of the link URL for your Slack channel.

Running this script with common keywords for keywords can flood the destination Slack channel with a large number of tweets, so it's a good idea to try it first with a test Slack channel.

Logging settings

I've included the logging setting for debugging, but you can delete it if you don't need it. If you execute it as it is, the following operation will be performed.

--The log file of twitter_monitoring.log is output on the same directory at the time of execution. --To output the log defined in logging.info, change the level parameter of logging.basicConfig in the code to logging.INFO in advance.

Script execution method

Execute the Python script by the following method.

Python library

Install the required Python library in advance.

sudo pip install --upgrade pip
sudo pip install slackclient
sudo pip install requests_oauthlib
sudo pip install logging

Execution command

Execute the python script with the following command.

nohup python twitter_monitoring.py &

By running it with nohup, the script will continue to run even if you log out of the Python executor.

that's all. After execution, make sure that the link of the tweet containing the character string specified in keyword will continue to be transferred in real time to the destination Slack channel.

Remarks

-Twitter API Terms of Service