[PYTHON] A story about deploying a Twitter-linked app created using Flask + gunicorn on Heroku

This is the first post. This article is a memorandum when deploying a Python app that automatically posts to Twitter using ** Flask + gunicorn ** on Heroku. Since I started from almost zero knowledge, I was addicted to various things, so I decided to write this article, hoping to help people in the same situation.

This paper has the following structure.

• Prerequisites
• Get Twitter API key
• Install various packages
• File structure
• Deploy Application (#アプリケーションのデプロイ)
• Bonus 1: Tweet with images
• Bonus # 2: Reply to reply tweet

Prerequisites

We will proceed on the assumption that the following is already installed.

• Python 3.7 series
• git

Get Twitter API key

Please refer to the following page to get API key & secret, Access token & secret. Summary of procedure from applying for Twitter API to approval in one shot (with example sentences) + API key, access token acquisition method In my case, the purpose of use was approved when I wrote the specifications of the bot in detail.

Installation of various packages

• requests: Rest API library for easy operation
• requests-oauthlib: A library for easy OAuth authentication in Python
• Flask: Python web framework
• gunicorn: The server that runs the WSGI app (Flask in this case)

Install with the following command.

command

pip install requests
pip install requests-oauthlib
pip install Flask
pip install gunicorn

file organization

The file structure under the working directory is as follows.

tree

test                        #Working directory
├── Procfile                #A file that describes commands to start on Heroku
├── requirements.txt        #File that describes the dependent libraries
├── runtime.txt             #A file that describes which version of Python to use on Heroku
├── config.py               #Authentication information
├── test_app.py             #App body
├── common_func.py          #Go out the function used in the app itself
└── json                    #Directory to put JSON files
    └── tweet_contents.json #A file that describes the content to be tweeted

tweet_contents.json

{
  "contents":
  {
    "img_url" : "Image URL",
    "tweet_text" : "Tweet sending test"
  }
}

Describe the body of the tweet in the Tweet transmission test of the above file. For image URL, specify the URL of the image you want to post. (Used in bonus 1) By the way, I prepared this image this time.

config.py

import os
from requests_oauthlib import OAuth1Session

CK = os.environ["CONSUMER_KEY"]          #Environment variable "CONSUMER_Set "KEY"
CS = os.environ["CONSUMER_SECRET"]       #Environment variable "CONSUMER_Set "SECRET"
AT = os.environ["ACCESS_TOKEN"]          #Environment variable "ACCESS_Set "TOKEN"
ATS = os.environ["ACCESS_TOKEN_SECRET"]  #Environment variable "ACCESS_TOKEN_Set "SECRET"
twitter = OAuth1Session(CK, CS, AT, ATS) #Authentication process

#end point
update_url = "https://api.twitter.com/1.1/statuses/update.json"       #Tweet sending endpoint
upload_media_url = "https://upload.twitter.com/1.1/media/upload.json" #Media upload endpoint

Credentials and endpoints are defined by global variables. Various API keys and access tokens required for authentication processing can be obtained from environment variables with os.environ ["environment variable name "]. By doing this, you do not have to modify the source code when the value changes due to token regeneration etc.

test_app.py

import os, json, common_func
from flask import Flask

app = Flask(__name__)

@app.route("/")
def tweet_test():
    common_func.post_tweet()
    return "Result OK."

if __name__ == "__main__":
    port = os.environ.get("PORT", 3333)
    app.run(host='0.0.0.0', port=port)

Describes the processing of the main body of the application. After starting the application, access the following URL using an appropriate web browser. http://0.0.0.0:3333/ When you access the above address, tweet_test () will be executed. If successful, Result OK. will be displayed on the page.

• If you cannot access the page with the above URL Depending on the browser, using 0.0.0.0 for the address may result in an error. In that case, replace 0.0.0.0 in the URL with localhost and try to access it. http://localhost:3333/

common_func.py

import os, json, config

def get_tweet_content(): #Get tweet body from json file
    json_file_path = os.path.join(os.path.dirname(__file__), "json/tweet_contents.json")
    try:
        f = open(json_file_path, "r", encoding="cp932")
    except FileNotFoundError as e:
        print(e)
        exit() #Exit if file not found
    json_data = json.load(f)
    return json_data["contents"]["tweet_text"]

def post_tweet(): 
    tweet_text = get_tweet_content()
    params = {"status": tweet_text}
    res = config.twitter.post(config.update_url, params=params)
    if res.status_code == 200: #If you can post normally
        print("Success.")
    else: #If you could not post normally
        res_text = json.loads(res.text)
        print("Failed. : %d, %s"% (res_text["errors"][0]["code"], res_text["errors"][0]["message"]))

• get_tweet_content()
  get_tweet_content () is a function that returns the tweet body read from the json file as a return value.
  Get the contents of the json file as an associative array with json.loads ().
• post_tweet()
  post_tweet () is a function for posting tweets.
  By setting params, you can specify the tweet content and reply destination.
  Specify the URL of the endpoint in the first argument of twitter.post () and the associative array param in the second argument.
  The execution result will look like the image.
  

Deploy application

Register Heroku and install Heroku CLI

Please refer to the following page to register Heroku and install Heroku CLI. [Heroku] Memo for deploying Python apps using Heroku on Windows [Python]

Creating a Heroku application

Log in to Heroku with the following command.

command

heroku login

Next, create an application with the following command. This time we will create an application with the name test_app.

command

heroku create test_app

After creating the application, move to the working directory, create a Git repository and make a remote connection. (This time, it is assumed that the working directory is created directly under the C drive)

command

cd /C/test
git init
heroku git:remote -a test_app

Creating a configuration file

Create the following configuration file.

• runtime.txt
  List the Python version.
  Please list the versions that Heroku supports.

runtime.txt

python-3.7.9

You can check the Python version of the local environment with the following command.

command

python --version

• requirements.txt
  Describes the module to be installed.

requirements.txt

Flask==1.1.2
gunicorn==20.0.4
requests==2.25.0
requests-oauthlib==1.3.0

You can check the list of modules installed in the local environment with the following command. All installed modules will be displayed, so list only the modules you need.

command

pip freeze

• Procfile
  Describes the command to be executed to start the application.

Procfile

web: gunicorn test_app:app --log-file -

Deploy

Execute the following command.

command

git add .
git commit -m "First deployment."
git push heroku master

Read the master in the command git push heroku master as appropriate for your environment.

trouble shooting

The causes of errors that occurred during deployment and their solutions are summarized below.

• buildpack error
  Error log: remote:! No default language could be detected for this app.
  Cause: Missing buildpack settings
  Solution: Execute the following command to set the build pack.
  Reference: Worried about heroku

command

heroku buildpacks:set heroku/python

• Procfile not found
  Error log: remote: No Procfile detected, using the default web server.
  Cause: The file name was mistakenly set to "Profile".
  Solution: Renamed the file to Procfile.
  (Supplement)
  Procfile has no ** extension **.
  If you create a file with a text editor, etc., it will be created in text format (.txt) unless otherwise specified, so be careful.
  It is safe to create a file with the following command.

command

echo "Start command" > Procfile

Setting environment variables

Set the environment variables used in the application with the following command. For the setting value, use the value obtained in "Getting Twitter API key".

command

heroku config:set CONSUMER_KEY="********"        # API key
heroku config:set CONSUMER_SECRET="********"     # API key secret
heroku config:set ACCESS_TOKEN="********"        # Access token
heroku config:set ACCESS_TOKEN_SECRET="********" # Access token secret

Confirm deployment

Access the Heroku app with the command below and check that the app is working properly.

command

heroku open

View log

You can display the log on the console with the following command. -t is an option to display the log in streaming state.

command

heroku logs -t

Bonus # 1: Tweet with an image

Modify common_func.py as follows.

common_func.py

import os, json, config, urllib.request

def get_tweet_content(): #Get tweet content from json file
    json_file_path = os.path.join(os.path.dirname(__file__), "json/tweet_contents.json")
    try:
        f = open(json_file_path, "r", encoding="cp932")
    except FileNotFoundError as e:
        print(e)
        exit() #Exit if file not found
    return json.load(f) #Return the object of the tweet content

def upload_media(img_url): #Upload the image to Twitter and media_Return id
    res = urllib.request.urlopen(img_url)
    img_data = res.read()
    img_files = {"media": img_data}
    res_media = config.twitter.post(config.upload_media_url, files=img_files)

    if res_media.status_code == 200:
        return json.loads(res_media.text)["media_id"]
    else:
        print("Image upload failed: %s", res_media.text)
        exit() #Exit if image upload fails

def post_tweet(): 
    tweet_content = get_tweet_content()
    media_id = upload_media(tweet_content["contents"]["img_url"])
    params = {"status": tweet_content["contents"]["tweet_text"], "media_ids" : media_id}
    res = config.twitter.post(config.update_url, params=params)
    if res.status_code == 200: #If you can post normally
        print("Success.")
    else: #If you could not post normally
        res_text = json.loads(res.text)
        print("Failed. : %d, %s"% (res_text["errors"][0]["code"], res_text["errors"][0]["message"]))

The execution result will look like the image.

Bonus # 2: Reply to reply tweets

Modify common_func.py as follows.

common_func.py

import os, json, config

def get_tweet_content(): #Get tweet body from json file
    json_file_path = os.path.join(os.path.dirname(__file__), "json/tweet_contents.json")
    try:
        f = open(json_file_path, "r", encoding="cp932")
    except FileNotFoundError as e:
        print(e)
        exit() #Exit if file not found
    json_data = json.load(f)
    return json_data["contents"]["tweet_text"]

def post_tweet(): 
    tweet_text = get_tweet_content()
    params = {"status": tweet_text, "in_reply_to_status_id" : "XXXXXXXXXXXXXXXXXXX"}
    res = config.twitter.post(config.update_url, params=params)
    if res.status_code == 200: #If you can post normally
        print("Success.")
    else: #If you could not post normally
        res_text = json.loads(res.text)
        print("Failed. : %d, %s"% (res_text["errors"][0]["code"], res_text["errors"][0]["message"]))

For in_reply_to_status_id, enter the ID of the tweet you want to reply to. The tweet ID is the 19-digit number listed at the end of the URL when the tweet is displayed on the browser version of twitter. https://twitter.com/username (screenname) /status/XXXXXXXXXXXXXXXXXXX

You cannot reply by simply specifying the ID of the tweet you want to reply to in in_reply_to_status_id. You need to add the user name (screen name) with @ at the beginning of the tweet body. Therefore, modify tweet_contents.json as follows.

tweet_contents.json

{
  "contents":
  {
    "img_url" : "Image URL",
    "tweet_text" : "@XXXXXXXX Reply test"
  }
}

Add @XXXXXXXXX to the beginning of tweet_text.

• In XXXXXXXX, enter the user name (screen name) of the account you want to reply to.

The execution result will look like the image.