The key to deploying Flask apps using Python Buildpack on Cloud Foundry (Diego)

About this article

For convenience, I touched Cloud Foundry (Diego) -based PaaS, so I tried deploying the app using Flask, which I usually use. Since it was my first time to touch CF seriously, there were some points that got caught, so I will describe it as a simple point. I hope it will be useful for those who are new to PaaS or who want to use it but are not familiar with it.

If you are familiar with CF, please tell me various things, such as pointing out the mistakes ...!

Sample code to learn about CF environment

I have put the sample code below. For those who are new to CF apps, comment on it. & Functions Super simple, but please forgive me.

This environment

I am trying using Enterprise Cloud PaaS of NTT Communications where Diego of Cloud Foundry is deployed. It seems that Diego has already deployed Pivotal Web Service, so I think it will work.

--Client used

How to move the sample code

Installation and configuration of cf_cli

First, install and set cf_cli to use Cloud Foundry. I think it is quick to see Official around here, but Max OS 64 bit, Windows 64 bit, The package for Linux 64 bit is distributed [^ 1], so just download and open it.

[^ 1]: While writing this article, I learned that Mac could also be done with Homebrew ...

After installation, log in with the cf command.

cf login [-a API_URL] [-u USERNAME] [-p PASSWORD] [-o ORG] [-s SPACE]

This is set using the account information provided by the service provider. Space and Org can be omitted if there is only one.

If you log in successfully, the following information will be returned.

--API endpoint and API version

API endpoint: https://paas-uk1-ecl.api.ntt.com (API version: 2.47.0)
User: ecid1*********0@econ1*********1
Org: econ1*********1
Space: demonstration

Once you log in, by default ~ / .cf / config.json is created and saved in your home directory. This can be relocated with the CF_HOME environment variable.

Get a list of users and check that you are logged in.

$ cf org-users econ1*********1 -a
Getting users in org econ1*********1 as 2*****************P8****nV...
USERS
2*****************P8****nV

Get sample code

Get the code from git

$ git clone [email protected]:yuta-hono/flask-cloudfoundry-sample.git

Execute cf push in the corresponding directory. This will deploy to CF with the current directory as an app. This time, the Python build pack is called from the external URL and CF community build pack with the -b option.

$ cf push <app name> -b https://github.com/cloudfoundry/python-buildpack

After a while, the build will be completed and the following result will be returned. If you are runnning, you are successful!

requested state: started
instances: 1/1
usage: 128M x 1 instances
urls: <yourapp>.uk1.eclpaas.com
last uploaded: Tue May 23 10:50:05 UTC 2016
stack: cflinuxfs2
buildpack: python_buildpack

     state     since                    cpu    memory          disk             details
#0   running   2016-05-24 10:50:52 PM   0.0%   18.8M of 128M   147.6M of 256M

Various things that are different from the ordinary environment you make yourself

Almost the original CF document asks questions and solutions that you want to put your application on PaaS but don't know what to look out for. -It is a quote from deploy.html # "Prepare to Deploy"), but I put it together in the above sample code. I haven't implemented all of them, but I've summarized the basic parts into simple code, so I think it will be easier to understand if you read it while looking at the sample code.

What should I do with the app's Listen port?

In CF, deploy-time ports are dynamically assigned in Container. It is possible to see the assigned port and IP, but the port and IP can be changed in that case because the application will be restarted in the event of a redeployment or other infrastructure or container failure. In other words, it cannot be described in Static.

CF is an [environment variable] called PORT inside the container where the app is deployed (http://docs.cloudfoundry.org/devguide/deploy-apps/environment-variable.html#PORT" Cloud Foundry Environment Variables It makes ") easier, so you need to dynamically set it as the app port.

In the sample code, it is written as follows. (Excerpt from hello.py)

hello.py


import os

~snip~

cf_port = int(os.getenv("PORT"))

~snip~

app.run(port=cf_port)

For reference, if you try deploying the sample code to CF, you can check the environment variable list of the deployment destination container at http: // <yourapp> / vars, so it is easy to understand if you check the environment variables provided by CF. I think.

How to see the app log

The application log can be obtained by outputting to the standard output STDOUT or the standard error output STDERR. (Make sure to output to either [http://docs.cloudfoundry.org/devguide/deploy-apps/streaming-logs.html#app) seems to be the key) In CF, there is a function to tail application logs in operation, and you can use this function to log. You can check it.

$ cf logs blue
Connected, tailing logs for app blue in org econ1*********1 / space demonstration as 2*1...

2016-05-24T22:51:21.02+0900 [RTR/0]      OUT demonstration.uk1.eclpaas.com - [24/05/2016:13:51:21 +0000] "GET / HTTP/1.1" 200 0 12 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36" 10.0.0.7:35858 x_forwarded_for:"192.168.0.43, 10.0.0.7" x_forwarded_proto:"http" vcap_request_id:8d5086b2-9ffd-4901-4ce5-80bcef39557c response_time:0.005617514 app_id:ae1b5d8f-2159-4259-947e-c4b01e1cd513
2016-05-24T22:51:21.02+0900 [APP/0]      ERR 10.0.50.151 - - [24/May/2016 13:51:21] "GET / HTTP/1.1" 200 -

There are multiple types of logs, the APP type is the application type, and the others are CF-provided logs. For more information, see Application Logging in Cloud Foundry.

It seems that it is necessary to consider a little more when operating it, but I would like to summarize it separately if there is an opportunity.

Please tell me who is more detailed

I want to use a scheduling job (cron)!

It doesn't seem to be a CF native feature. It seems that some service providers that provide CF implement their own task scheduler as an external module, but according to Pivotal, a big developer of CF, [If you do not have Cron, you can run the job as an app. Not good](https://blog.pivotal.io/labs/labs/scheduling-tasks-on-cloud-foundry "Scheduling tasks on Cloud Foundry" ") (A lot of translation).

I want to set environment variables myself, can I?

can. In the sample code, .profile.d is the directory for this. By placing a shell script with the .sh extension inside this directory, this script will be executed on the CF side at the time of deployment, so you will use it to set environment variables.

As an example, in this sample code, MY_OWN_ENV_VAR is set, otherwise the environment variables provided by CF will exist.

For reference, in the case of the sample code app, you can see that it can be set at http: // <yourapp> / vars. Alternatively, you can log in to the container where the app is deployed with $ cf ssh <yourapp>, so after logging in, you can check the list of environment variables with $ printenv.

Can I spit out the data from the app and put it locally?

**Is useless. ** ** In the CF environment, the data placed inside the container to which the application is deployed is volatile except for the data included at the time of deployment. So it disappears easily when the app is restarted in another container. Persistent data will use external storage. Static ones will use s3 object storage and other storage, and data is likely to use an external RDB service. Depending on the provider, there are also some providers that are integrated and provided, so please check before using.

I will omit the specific usage method because it will be a little more complicated than the sample code, but Bind and use services ), It seems that you can easily do this.

I want to include files in the repository that I want to ignore during deployment

You want to ignore README.md and .git and below. In such a case, use .cfignore.

This is almost the same mechanism as .gitignore. By default, it will ignore the following files and directories (https://docs.cloudfoundry.org/devguide/deploy-apps/prepare-to-deploy.html#exclude).

The sample code is set to ignore README.md and LICENSE. As an example, if you try cf ssh <app_name> in the deployed application,

vcap@gisruph1uo0:~$ ls
app  logs  staging_info.yml  tmp

It looks like this. ʻApp` There is an app deployed inside, so let's take a look.

vcap@gisruph1uo0:~$ cd app
vcap@gisruph1uo0:~/app$ ls
hello.py  Procfile  requirements.txt  runtime.txt  templates

It is properly ignored. You can try the effect by editing .cfignore, so please use it.

I want to restrict access on the network

In terms of network, it will be listened to 0.0.0.0/0. Also, the binding port of the application itself will be listened to 0.0.0.0/0. So network access restrictions are (almost) unavailable.

However, since the client IP is included in the HTTP request header called X-Forwarded-For on the CF side, it is possible to restrict access by the source IP on the application side by using this. (X-Forwarded-For is familiar in AWS ELB etc.)

In the sample code, it is done as follows. (Excerpt / edit only the relevant part)

from flask import request

srcIp = request.access_route[0]

# List of IP addresses to allow the access
allowed_ip = ['192.0.2.1', '192.0.2.2']

@app.route('/ip')
def showIp():
if srcIp not in allowed_ip:
    abort(403)

    return 'Your IP %s is allowed' % srcIp

I can get X-Forwarded-For with flask.request.access_route, but Flask stores the value added to X-Forwarded-For as an array. The ʻaccess_route [0]specified here is the first one when the value ofX-Forwarded-For is 192.0.2.1, 192.0.2.2, that is, 192.0.2.1`. It will get you a feeling.

In terms of CF, the client source IP is only put in X-Forwarded-For, but depending on the provider, a load balancer etc. may be sandwiched in front and multiple values may be stored, so Adjustment may be required in some cases.

In the sample code, access restriction by client source IP is implemented at http: // <yourapp> / ip. You can change the allowed IP by changing the list of ʻallowed_ip`, so please try it.

Supplement (Flask story)

In the sample code, the method is defined for the / ip resource to judge the access permission for simplicity, but in Flask, the operation before accepting the request is set in the decorator for before_request. Yes, you can simply restrict access to all pages. See Flask Official Documentation, API Chapter for more information.

In the sample code environment, you can see the HTTP request header at the bottom of the screen at http: // <yourapp> / vars.

Finally

This time, I tried to put together the application by deploying it on CF for the time being and running it. When it comes to the operation stage (deployment, log collection, monitoring, etc.), I think that there will be many issues again, but since it will be long, I would like to summarize it in another article if there is an opportunity.

Reference material

--Cloud Foundry Official Document (http://docs.cloudfoundry.org/)

Recommended Posts

The key to deploying Flask apps using Python Buildpack on Cloud Foundry (Diego)
[Heroku] Memo for deploying Python apps using Heroku on Windows [Python]
The road to installing Python and Flask on an offline PC
How to update the python version of Cloud Shell on GCP
Work memo to migrate and update Python 2 series scripts on the cloud to 3 series
[IBM Cloud] I tried to access the Db2 on Cloud table from Cloud Funtions (python)
Convert the cURL API to a Python script (using IBM Cloud object storage)
Python: Try using the UI on Pythonista 3 on iPad
Run Python web apps on NGINX + NGINX Unit + Flask
Introduction to Python with Atom (on the way)
Sound the buzzer using python on Raspberry Pi 3!
Run the flask app on Cloud9 and Apache Httpd
Think about how to program Python on the iPad
Steps to install the latest Python on your Mac
From python to running instance on google cloud platform
Write data to KINTONE using the Python requests module
Post to your account using the API on Twitter
[Introduction to Python] How to stop the loop using break?
Output to "7-segment LED" using python on Raspberry Pi 3!
How to enjoy Python on Android !! Programming on the go !!
Use Python to monitor Windows and Mac and collect information on the apps you are working on
How to know the number of GPUs from python ~ Notes on using multiprocessing with pytorch ~
Control smart light "Yeelight" from Python without using the cloud
[Python3] Format the character string using the variable name as the key.
[Python] Change the Cache-Control of the object uploaded to Cloud Storage
[python] How to check if the Key exists in the dictionary
[Hyperledger Iroha] Notes on how to use the Python SDK
Sample to put Python Flask web app on Azure App Service (Web App)
View using the python module of Nifty Cloud mobile backend
How to deploy the easiest python textbook pybot on Heroku
Connect to the Bitcoin network using pycoin (Python Cryptocoin Utili)
Get Python scripts to run quickly in Cloud Run using responder
Save images on the web to Drive with Python (Colab)
Touch NoSQL with Python using the Oracle NoSQL Database Cloud Simulator
Tweet Now Playing to Twitter using the Spotify API. [Python]
I tried changing the python script from 2.7.11 to 3.6.0 on windows10
[Python] I tried to make a simple program that works on the command line using argparse.