Create a "Congratulations 2.0" app that collects congratulatory voices with automatic phone answering with Twilio API x Python (Flask) x Heroku
Introduction
We have created a "Congratulations 2.0" app that secretly collects the "voices" of "Congratulations on your marriage" from your friends and presents them to the bride and groom. The original story is a nice blog here in 2013
-[Congratulations on your marriage] Code and design story given to your best friend](http://mamipeko.hatenablog.com/entry/happy-wedding-s)
If you haven't read it, please read it. I think "programmers are really cool". I was deeply moved by this blog, so I made a complete copy of this content.
To briefly summarize the contents of the blog,
--Create a phone app for auto-answer messages using a voice API called Twilio --Have the bride and groom friends call the designated phone number and record the "Congratulations on your marriage" message. --A gift with a congratulatory message on your iPod, etc.
The difference between the content of the blog and this time is
--Original story: Implemented in Ruby → This time: Implemented in Python (Flask) --Original story: Create a nice GUI page to handle the data acquired by twilio API → This time: Data management (cut out) on the management screen page of twilio API without creating GUI
Although the above blog and the blog of the engineer of the planning member mention the processing flow, the complete code of the working form was not published, so I will refer to the contents of the blog and do the same operation. It feels like it was implemented in Python. The fragmentary script was published on the blog by the engineer members.
-Collecting congratulatory messages on Twilio for # wedding-s
I usually use Python to analyze data, so I've never used a web framework like Flask to create something like an app, or a hosting service like Heroku. was.
That's why I will leave a note about the various moss of the web application (?) Amateur and the points that are not explained in detail in the Twilio API official document.
Register with Twilio API
The following blogs were helpful for registering Twilio, billing methods, purchasing phone numbers, installing static sample files on S3, and checking their behavior. This time we will not handle static files, so it is ok to go through. It's okay if you complete the registration to the API for the time being.
-I checked the basic usage of the telephone API Twilio
Twilio is a paid API service (no free tier), which first charges 2000 yen as the minimum amount, and then charges an additional 1000 yen. One call costs about 0.1 to 2 yen, so I think the first 2000 yen is enough for simple application development.
By the way, if you are asking "What is S3?", This page will be helpful.
-AWS Reintroduction Amazon S3 Edition
S3 is a cloud file server operated by Amazon. The first credit card registration is a hassle, but it's very cheap and easy to use, so use it. (S3 will be a place for voice data for automatic directory assistance)
Creating a local environment
-[How to set up PYTHON and Flask development environment](https://jp.twilio.com/docs/guides/how-to-set-up-your-python-and-Flask-development-environment#choose-a- python-version)
Follow the page above to set up your local development environment. The main thing to do is the following installation.
--Install python --Install virtualenv --Installing Flask --Installing the Twilio Python SDK --Installing ngrok (described later)
vertualenv builds a virtual environment. When hosting a set of web application scripts on the Heroku server, separate the environment for this application with verticalenv so that unnecessary modules etc. are not mixed in. Install it so that it will not be troublesome later (see below if you want to create a virtual environment with the anaconda package) Details of ngrok will be described later.
Understanding the Twilio API
-VOICE TWIML PYTHON Quick Start
After setting up the development environment, take a look at the contents of the quick start above to see what kind of task you can actually write with the Twilio API. (Because I'm the type that I can't understand unless I actually move it and see how it behaves ...)
When you receive a call, Twilio will come to see TwiML (XML-like) at the specified URL and will process the format defined there at any time.
If you look at the above tutorial, you can see how to write TwiML and how to generate it dynamically, but there are some old descriptions (descriptions that do not match the latest API specifications), and Flask scripts are put on the essential web server. The hosting method is unfriendly, so I get a lot of moss. .. .. In this tutorial, I think it's enough to confirm that "How to write Twilio API is like this".
Among the functions of the Twilio API, the application created this time mainly handles incoming calls, such as "accepting calls", "playing recorded audio files", "recording messages from users", and "accepting number input". I am using.
The actual application sequence is as follows (quoted from hmsk's blog)
-(1) Accept the call and send a welcome message -(2) Show the explanation of the message input method (like pressing # when finished) -(3) Sound the dial tone and record (end with #) -(4) Declare to play the recorded message and play it -(5) If you like this message, press "1", and if you want to record again, press "3" and wait for input. -(6) If the entered number is "1", redirect to (7), if it is "3", redirect to (2), otherwise redirect to (5). -(7) Thank you and hang up
The following is a tutorial for each function, but as I wrote above, please note that there are some parts that do not work even if you write according to the tutorial.
-PYTHON Quick Start: Read Message -PYTHON Quick Start: Play MP3 files to callers -PYTHON Quick Start: Record a message from the caller
Below is the completed Flask script that I wrote while referring to the above sequence, tutorial, and how to use the latest API.
(The mp3 file called in the script is the voice data for automatic response installed in S3.)
Execution in local environment (how to use ngrok)
Next, test locally whether the created web application (Flask script) works properly.
Since it is troublesome, I want to test the behavior without deploying (uploading to the server), but in that case ngrok is convenient. It's free so let's use it.
Download the ngrok that matches your OS from the Official Page. On Mac, brew install ngrok is ok
Unzip the DL file
$ unzip /path/to/ngrok.zip
After that, if the display is confirmed with ./negrok help etc., it is ok
In order to actually use ngrok, you need to register as a user by signing up from the ngrok official page (free of charge).
After registering as a user and logging in, execute ./ngrok authtoken (followed by a character string) displayed on the Get Started screen in the terminal. I will leave it
(Maybe this user registration and auth token registration is not mandatory)
Once you've completed all the ngrok operations, run the .py Flask script you want to run. (When executed, it will be ready to be displayed on localhost: 5000)
With Flask running, run ./ngrok http 5000 in a separate window.
Then, on the newly launched screen
Forwarding http://6813xaa9.ngrok.io -> localhost:5000
Forwarding https://6813xaa9.ngrok.io -> localhost:5000
Since something like this is written, register the displayed URL (either http or https) in the "Webhook" section of the twilio API management screen and save it. What you do locally will now be treated as if it were running on your web server.
After that, when you call the phone number, the process written in Flask should be executed.
By the way, ngrok returns a random url every time it starts. If you want to fix this, please refer to the item "Random character strings are troublesome" on this page. (* However, as of April 2017, you may have to become a paid member in order to obtain a fixed URL?)
Hosting to a web server (how to use Heroku)
If it's a python script that creates a TwiML for static behavior (for example, just autoplays the specified mp3 when you get a call), put it in S3 and put the URL of the script on the Twilio API admin page You can do it by registering with.
However, if you want to perform complicated processing (like this time) such as changing the processing according to the number entered by the user, install a python script on an external server and dynamically TwiML according to the input contents of the user. Must be generated.
As an external server option to install the script, EC2 of Amazon Web Service (AWS) and Google App Engine (GAE) of Google Cloud Platform (GCP) can be used at a low price, but this time the target process is simple For that reason, I decided to use Heroku, which is easy to install and can be used for free. Heroku is a service that allows you to run apps by simply pushing a set of git-managed web scripts to a designated Heroku server.
For the procedure to run a web application (Django or Flask) written in Python on Heroku, I referred to the official tutorial below.
- Getting Started on Heroku with Python -(There was also a Japanese version of the above content in Qiita)
The general flow of the tutorial is
--Create a Heroku account
--DL for Heroku Command Line Interface (CLI)
--Login to heroku using CLI
--Create a repository for your app on Heroku with heroku create
--Push the script to the created repository with git push heroku master
--Launch the app with heroku open
It's like that. Easy. By the way, since the DB system operation is not performed in the application created this time, all the installation processes such as SQLAlchemy and psycopg2 are skipped.
The minimum files you need to push to Heroku to run the app are:
--main.py ・ ・ ・ Main body file in which Flask script is written (here, the file name is only main.py) --Procfile ・ ・ ・ File in which commands for executing main.py are written (details will be described later) --requirements.txt ・ ・ ・ List file of modules imported by main.py
(Caution) The following processing is performed after entering the virtual environment separated for the application by verticalenv or anaconda.
Register for a Heroku account http://www.heroku.com/
Install the Heroku CLI tool for your OS from here.
After user registration and CLI installation, heroku login in the terminal
% heroku login
#->Enter your heroku ID password
Put the directory containing the python script (Flask script) you want to deploy on the Heroku server under git control
$ cd /path/to/dir
$ git init
Under the same directory, create requirements.txt and Procfile first.
$ pip freeze > requirements.txt
(If you don't have gunicorn installed, pip install gunicorn)
(Match the "main" part with the Flask file name)
$ touch Procfile
$ echo web: gunicorn main:app --log-file - >> Procfile
heroku create in the (same) directory that was git init
$ heroku create <Any app name> --buildpack heroku/python
heroku create will issue a Heroku URL
Register this URL in the webook section of the twilio API console (note that it's easy to forget)
Push the script to heroku
$ git add -A
$ git commit -m “hogehoge”
$ git push heroku master
Rough letters flow It is pushed to master safely, and if there is no error, it is ok
$ heroku open
Then, if the pushed content is correct, the app should be displayed in the browser.
If it worked locally but it didn't work,
--Are there any modules that have not been added to requirements.txt? --Is the Procfile written correctly?
(Mostly there is something wrong with the Procfile)
The above series of contents was neatly organized in This Qiita article and was helpful.
About Procfile
It seems that the execution command of the application pushed to Heroku should be written in what is called Procfile (Details about Procfile are written in here However, even if you don't understand it, you can write it as follows).
When writing a Procfile using something called gunicorn, I tried to imitate this blog.
For example, if you want to run a Flask script named main.py on Heroku, the Procfile will
It seems that you only need to write and save the line web: gunicorn main: app --log-file -. (Not python main.py.)
(Gunicorn seems to be recommended to use this in various articles, so I'm not really sure, but I use it for the time being. If you do not use gunicorn, Flask and django on port 5000 There seems to be a problem when you stand up)
Check the apps pushed to Heroku
Check which apps are currently being pushed to Heroku (up to 5 in the free tier)
$ heroku apps
Delete unnecessary apps
$ heroku destroy --app <app name>
#->Type the app name for confirmation again
Check the script currently running on heroku's server (including the remaining time that can be started in the free frame)
$ heroku ps
Check log
$ heroku logs #— You can see the log tail with the tail option
See here for other Heroku CLIs
Other
Can't answer calls from overseas?
On the way, I found out that the phone number I purchased from Twilio couldn't be answered from overseas, so I ended up having a friend living overseas send me the sound source recorded by the voice recorder app. The meaning of making it a phone app. .. .. Perhaps the phone number I purchased was limited to Japan? (I have not investigated in detail)
Trimming mp3 files
The user's message data recorded by the API is at the login destination from https://jp.twilio.com/login/kddi-web. You can do DL etc. from the console. (I haven't checked how many messages can be saved, but in my case I confirmed that 60 messages were saved.)
I think you could choose mp3 or WAV as the data storage format. In my case, I used to download audio data with mp3. If you want to cut out the beginning part and the butt part of the voice data and edit it easily, this is free and convenient. You can easily trim the audio on the browser (no particular maximum number of files).
Play verb and m4a file problem
Note that the m4a file output by the iPhone voice recorder app as standard cannot be played with the play verb of the twilio API! (Mp3 is safe)
--Reference: https://jp.twilio.com/docs/api/twiml/play
Create a virtual environment with anaconda
For those who want to turn off the environment with anaconda instead of vertualenv (usually I think you should use verticalenv)
Here's how to create an env with anaconda and get in and out of it
--Reference: Building a python environment for those who aim to become data scientists 2016
Building a virtual environment
$ conda create -n <Environment name> python=<version> <Write the libraries you want to install separated by spaces>
Example: conda create -n py_ver2 python = 2.7 numpy scipy pandas jupyter
It is also possible to create a new environment by collecting anaconda environments (creating an anaconda virtual environment with anaconda)
$ conda create -n anaconda2 python=2.7 anaconda
Display a list of virtual environments created with the currently existing conda
$ conda env list
Enter the virtual environment
$ source activate <Environment name>
Get out of the virtual environment
$ source deactivate
reference
Pages referenced other than the above links
-List of user request status codes generated by Twilio API -Twilio API Alert Trigger List
Digression
I wrote the background of making this app on my personal blog. If you are interested.
Recommended Posts